Elamur Research⌕
☰
home / papers / 2018 / Foreign English Accent Adjustment by Learning Phonetic Patterns

Foreign English Accent Adjustment by Learning Phonetic Patterns

Paper · 2018 · Fedor Kitashov, Elizaveta Svitanko and Debojyoti Dutta

Summary

Speech recognizers perform poorly on rare foreign accents because there is little accented training data. The authors aim to learn accent patterns automatically, instead of relying on rules written by hand by phonologists. They use the Speech Accent Archive (GMU), where speakers from many language backgrounds read the same "Please call Stella" paragraph. They align IPA transcriptions of these readings with General American English (GAE) pronunciations. For every sound, they count how often it is replaced, deleted, or has another sound inserted before or after it. These counts become probabilities, which they apply to the CMU Pronouncing Dictionary to synthesize about a million accented word transcriptions. On the detailed GMU symbol set, the statistics recover all 20 expert-listed generalizations. On the reduced CMU inventory they recover 13. A character-level seq2seq LSTM is then trained to map Russian-accented transcriptions back to standard ones, in the manner of a spell-checker, and reaches 59% accuracy. The study shows that a small set of expert transcriptions can be expanded into large synthetic training data.

Key points

  • Source data is the GMU Speech Accent Archive: 2511 recordings covering 239 ethnology codes, about half of them transcribed in IPA. The data covers 69 words of the elicitation paragraph.
  • A reduction dictionary collapses 169 distinct GMU transcription characters into the 39 sounds used by the CMU Pronouncing Dictionary.
  • Per-sound statistics include total count, replacements, deletions, and insertions before and after the sound. Multi-sound substitutions are allowed, such as [p] becoming [b@].
  • Learned generalizations match 20/20 expert features on the full GMU symbols but only 13/20 on the simplified CMU set. The CMU dictionary grows from about 103,000 single-accent entries to about one million multi-accent samples.
  • The correction model is an encoder-decoder LSTM with 256 units, trained with RMSprop, batch size 4096, for 100 epochs. Training uses about 800,000 Russian-accented samples split 80/10/10. Words up to 14 characters cover 99% of the corpus. Reported accuracy is 59%.

Interesting ideas

  • I1. Learn a per-symbol edit model (substitute, delete, insert left or right) from a handful of aligned variant/standard pairs, then sample from it to produce realistic noisy variants at scale; this is cheap and easy to interpret.
  • I2. Treat accent normalization as spelling correction, and train the corrector only on synthetic noise derived from a clean dictionary, because the small real dataset just gets memorized.
  • I3. Map a rich annotation inventory onto a coarser one, then measure explicitly which expert-known distinctions survive (20 versus 13). This makes the cost of simplification visible rather than hidden.
  • I4. Validate automatically induced rules by checking them against an independent list of rules that experts compiled by hand.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Counts learned from real misspellings or nonstandard spellings could generate noisy word forms for stress-testing the tokenizer, tagger and lemmatizer, with each variant traceable to explicit edit statistics.
  • I2 (learn): Silver training data for a normalizer could be built by corrupting clean lexicon entries with learned edit probabilities, keeping real samples only for evaluation.
  • I3 (tree): When converting between UD dialects or tagsets, a checklist of linguistically known distinctions could show which ones a conversion keeps and which it erases.
  • I4 (learn): Self-induced repair rules could be scored on how many hand-written grammar facts they rediscover, giving an explainable sanity check alongside accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 worksSignal Processing · 213 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Fedor Kitashov, Elizaveta Svitanko and Debojyoti Dutta. 2018. Foreign English Accent Adjustment by Learning Phonetic Patterns. arXiv:1807.03625.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1807.03625 · arXiv 1807.03625

@misc{kitashov2018foreign,
  title = {{Foreign English Accent Adjustment by Learning Phonetic Patterns}},
  author = {Fedor Kitashov, Elizaveta Svitanko and Debojyoti Dutta},
  year = {2018},
  doi = {10.48550/arxiv.1807.03625},
  eprint = {1807.03625},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1807.03625}
}

References

18 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Jan Chorowski, Dmitriy Serdyuk, Philemon Brakel, and Yoshua Bengio. 2015. End-to-end attention-based large vocabulary speech recognition. CoRR abs/1508.04395. http://arxiv.org/abs/1508.04395. ↗
  2. William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. 2015. Listen, attend and spell. CoRR abs/1508.01211. http://arxiv.org/abs/1508.01211. ↗
  3. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statis CoRR abs/1406.1078. http://arxiv.org/abs/1406.1078. ↗
  4. Cisco Systems. 2018. Reduction dictionary, statistical and seq2seq models. https://github.com/CiscoAI/accent_transfer. ↗
  5. G. E. Dahl, D. Yu, L. Deng, and A. Acero. 2012. Context-dependent pre-trained deep neural networks for large-vocabu IEEE Transactions on Audio, Speech, and Language Processing 20(1):30–42. https://doi.org/10.1109/TASL.2011.2134090. ↗
  6. Shaona Ghosh and Per Ola Kristensson. 2017. Neural networks for text correction and completion in keyboard decod CoRR abs/1709.06429. http://arxiv.org/abs/1709.06429. ↗
  7. Alex Graves, Abdel-rahman Mohamed, and Geoffrey E. Hinton. 2013. Speech recognition with deep recurrent neural networks. CoRR abs/1303.5778. http://arxiv.org/abs/1303.5778. ↗
  8. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine 29(6):82–97. ↗
  9. Xuedong D Huang, Yasuo Ariki, and Mervyn A Jack. 1990. Hidden markov models for speech recognition . find
  10. International Phonetic Association. 2005. International Phonetic Alphabet. http://www.langsci.ucl.ac.uk/ipa/ipachart.html. ↗
  11. Stephen A. Kunath and Steven H. Weinberger. 2009. Stat: Speech transcription analysis tool. find
  12. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session. Association for Computational Linguistics, Stroudsburg, PA, USA, NAACL-Demonstrations ’09, pages 9–12. http://dl.acm.org/citation.cfm?id=1620959.1620962. ↗
  13. Abdel-rahman Mohamed, George E Dahl, and Geoffrey Hinton. 2012. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing 20(1):14–22. ↗
  14. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. 2011. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding. IEEE Signal Processing Society, EPFL-CONF-192584. find
  15. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. CoRR abs/1409.3215. http://arxiv.org/abs/1409.3215. ↗
  16. The Speech Group. 2014. The CMU Pronouncing Dictionary. http://www.speech.cs.cmu.edu/cgi-bin/cmudict. ↗
  17. Steven Weinberger. 2015. Speech Accent Archive. George Mason University. http://accent.gmu.edu. ↗
  18. X. Yang, K. Audhkhasi, A. Rosenberg, S. Thomas, B. Ramabhadran, and M. Hasegawa-Johnson. 2018. Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition. ArXiv e-prints .

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kitashov-2018-foreign-english-accent-adjustment-learning/