Elamur Research⌕
☰
home / papers / 2021 / IE-CPS Lexicon: An Automatic Speech Recognition Oriented Indian-English Pronunciation Dictionary

IE-CPS Lexicon: An Automatic Speech Recognition Oriented Indian-English Pronunciation Dictionary

Paper · 2021 · Shelly Jain, Aditya Yadavalli, Ganesh Mirishkar et al.

Summary

ASR systems built for American or British English perform poorly on Indian English (IE). The main reason is that Indian speakers, influenced by their native languages, systematically pronounce many English sounds differently. The authors address this with IE-CPS, a pronunciation lexicon that rewrites every phone of CMUdict into the Common Phone Set (CPS), an established shared phone inventory for six major Indian languages.

The mappings follow perceptual similarity and cover only sound changes that appear across most Indian language regions. Their own analysis shows that substitutions far outnumber insertions and deletions in IE speech. The lexicon therefore contains substitution rules only, which removes the need for a letter-to-phoneme aligner.

They train an IE ASR system on the NPTEL corpus and contrast it with an American system trained on Librispeech. Using IE-CPS instead of CMUdict cuts WER by up to 3.95%. This matters because the lexicon is rule-based and needs no training data. It drops straight into existing hybrid pipelines and fits code-switched settings where the other language already uses CPS.

Key points

  • IE-CPS maps all ARPAbet-based CMUdict phones onto CPS (Ramani et al., 2013), which covers Hindi, Marathi, Bengali, Tamil, Malayalam and Telugu. It extends the mapping of Ganji et al. (2019) but reuses existing CPS codes (e.g. for AA and AO) rather than adding new ones.
  • There are four rule families:
    • dental fricatives become dental plosives, and alveolar plosives become retroflex plosives;
    • diphthongs become long vowels (except [aU] and [aI]);
    • [v] and [w] merge;
    • vowels shift toward the nearest native-language vowel.
  • In the authors' analysis, deletions occur below 1%, insertions below 15% and substitutions above 30%. Region-specific insertions, such as Hindi vowel epenthesis, are deliberately left out.
  • Compared with CMUdict, IE-CPS gives a WER reduction of up to 3.95% on NPTEL, a corpus with a broad range of regional accents. In one example, CMUdict misrecognises "disappointing" as "This appointing", while IE-CPS gets it right.
  • The authors argue that hybrid ASR with a rule-based lexicon beats E2E and seq2seq G2P when the accent is low-resource. Per-region lexicons and code-switched modelling are listed as future work.

Interesting ideas

  • I1. Measure which error type dominates (substitution vs insertion/deletion) and model only that one, so a whole alignment module can be dropped.
  • I2. Encode only the changes shared by most varieties in a common core, and leave region-specific behaviour to thin add-on rule layers.
  • I3. Reuse an already standardised inventory instead of inventing new codes, so resources for different languages share one vocabulary and combine easily.
  • I4. Make the new component a drop-in replacement for the baseline (lexicon swap only, rest of pipeline fixed), so the measured gain is attributable to that one change.
  • I5. Hand-written rules need no training data and do not degrade on the rare long tail the way learned seq2seq models can.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Non-standard or learner spellings could first be profiled the same way, and a substitution-only normalizer would keep lemma lookup simple and explainable.
  • I2 (tree): UD dialect conversion between treebanks could use one shared conversion core plus small per-treebank patches, making each difference easy to inspect.
  • I3 (world): Compiled knowledge should attach to existing WordNet/FrameNet/Wikidata identifiers rather than fresh in-house IDs, so new sources can be merged without remapping.
  • I4 (learn): New rule modules or repair rules could be evaluated by swapping them into an otherwise frozen pipeline against the old version as a control.
  • I5 (tokens): Morphology and lemma rules can cover rare word forms where the tagger's statistics are thin, with the learned model handling frequent cases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Shelly Jain, Aditya Yadavalli, Ganesh Mirishkar, Chiranjeevi Yarra and Anil Kumar Vuppala. 2021. IE-CPS Lexicon: An Automatic Speech Recognition Oriented Indian-English Pronunciation Dictionary. Proceedings of the 18th International Conference on Natural Language Processing (ICON).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{jain2021ie,
  title = {{IE-CPS Lexicon: An Automatic Speech Recognition Oriented Indian-English Pronunciation Dictionary}},
  author = {Shelly Jain, Aditya Yadavalli, Ganesh Mirishkar, Chiranjeevi Yarra and Anil Kumar Vuppala},
  booktitle = {Proceedings of the 18th International Conference on Natural Language Processing (ICON)},
  year = {2021},
  url = {https://aclanthology.org/2021.icon-main.24/}
}

References

33 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aryaman Arora, Luke Gessler, and Nathan Schneider. 2020. Supervised grapheme-to-phoneme conversion of orthographic schwas in hindi and punjabi. ArXiv, abs/2004.10353. ↗
  2. R. K. Bansal. 1969. The intelligibility of Indian English: measurements of the intelligibility of connected speech, and sentence and word material, presented to listeners of different nationalities. Central Institute of English. find
  3. Alan W Black, Kevin Lenzo, and Vincent Pagel. 1998. Issues in building general letter to sound rules. In The third ESCA/COCOSDA workshop (ETRW) on speech synthesis. find
  4. Stanley F Chen and Joshua Goodman. 1999. An empirical study of smoothing techniques for language modeling. Computer Speech & Language, 13(4):359–394. ↗
  5. Najim Dehak, Patrick J Kenny, Réda Dehak, Pierre Dumouchel, and Pierre Ouellet. 2010. Front-end factor analysis for speaker verification. IEEE Transactions on Audio, Speech, and Language Processing, 19(4):788–798. ↗
  6. Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah, Ankita Singh, Srinivasa Raghavan K. M., Shreya Khare, Vinit Unni, Saurabh Vyas, Akash Rajpuria, Chiranjeevi Yarra, Ashish R. Mittal, Prasanta Kumar Ghosh, Preethi Jyothi, Kalika Bali, Vivek Seshadri, Sunayana Sitaram, Samarth Bharadwaj, Jai Nanavati, Raoul Nanavati, Karthik Sankaranarayanan, Tejaswi Seeram, and Basil Abraham. 2021. Multilingual and code-switching ASR challenges for low resource indian languages. CoRR, abs/2104.00235. ↗
  7. Emmanuel Ferragne and François Pellegrino. 2004. A comparative account of the suprasegmental and rhythmic features of british english dialects. Mod-elisations pour l’Identification des Langues. find
  8. Mark JF Gales. 1998. Maximum likelihood linear transformations for hmm-based speech recognition. Computer speech & language, 12(2):75–98. ↗
  9. Sreeram Ganji, Kunal Dhawan, and Rohit Sinha. 2019. Iitg-hingcos corpus: A hinglish code-switching database for automatic speech recognition. Speech Communication, 110:76–89. ↗
  10. Kyle Gorman, Lucas F.E. Ashby, Aaron Goyzueta, Arya McCarthy, Shijie Wu, and Daniel You. 2020. find
  11. The SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion. In Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 40–50, Online. Association for Computational Linguistics. find
  12. Xian Huang, Xin Jin, Qike Li, and Keliang Zhang. 2020. On construction of the asr-oriented indian english pronunciation dictionary. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 6593–6598. find
  13. Abhinav Jain, Minali Upreti, and Preethi Jyothi. 2018. Improved accented speech recognition using accent embeddings and multi-task learning. In Interspeech, pages 2454–2458. find
  14. Sunhee Kim, Kyuwhan Lee, and Minhwa Chung. 2011. A corpus-based study of english pronunciation variations. In Twelfth Annual Conference of the International Speech Communication Association. find
  15. Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In Proc. Interspeech. find
  16. Sreeja Manghat, Sreeram Manghat, and Tanja Schultz. 2020. Malayalam-English Code-Switched: Grapheme to Phoneme System. In Proc. Interspeech 2020, pages 4133–4137. find
  17. Peggy Mohan. 2021. WANDERERS, KINGS, MER-CHANTS: the story of india through its languages. PENGUIN. find
  18. David R Mortensen, Siddharth Dalmia, and Patrick Littell. 2018. Epitran: Precision g2p for many languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
  19. S. Nair, Cr Rechitha, and C. S. Kumar. 2013. Rule-based grapheme to phoneme converter for malayalam. find
  20. Raymond WM Ng and Keikichi Hirose. 2012. Syllable: A self-contained unit to model pronunciation variation. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4457–4460. IEEE. find
  21. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210. find
  22. Alok Parlikar, Sunayana Sitaram, Andrew Wilkinson, and Alan W Black. 2016. The festvox indic frontend for grapheme to phoneme conversion. In WILDRE: Workshop on Indian Language Data-Resources and Evaluation. find
  23. Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. A time delay neural network architecture for efficient modeling of long temporal contexts. In Sixteenth annual conference of the international speech communication association. find
  24. Ben Peters, Jon Dehdari, and Josef van Genabith. 2017. Massively multilingual neural grapheme-to-phoneme conversion. arXiv preprint arXiv:1708.01464. ↗
  25. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. 2011. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding, CONF. IEEE Signal Processing Society. find
  26. Daniel Povey, Vijayaditya Peddinti, Daniel Galvez, Pegah Ghahremani, Vimal Manohar, Xingyu Na, Yiming Wang, and Sanjeev Khudanpur. 2016. Purely sequence-trained neural networks for asr based on lattice-free mmi. In Proc. Interspeech, pages 2751– 2755. find
  27. B Ramani, S Lilly Christina, G Anushiya Rachel, V Sherlin Solomi, Mahesh Kumar Nandwana, Anusha Prakash, S Aswin Shanmugam, Raghava Krishnan, S Kishore Prahalad, K Samudravijaya, et al. 2013. A common attribute based unified hts framework for speech synthesis in indian languages. In Eighth ISCA Workshop on Speech Synthesis. find
  28. Marco Schilk. 2010. Pingali sailaja, indian english (dialects of english). edinburgh: Edinburgh university press, 2009. pp x 172. hardback £60.00, isbn 978-0-7486-2594-9, paperback £19.99, isbn 978-0-7486-2595-6. English Language and Linguistics, 14(1):135–139. find
  29. Andreas Stolcke. 2002. Srilm-an extensible language modeling toolkit. In Proc. Seventh International Conference on Spoken language Processing. find
  30. A. Wasala, R. Weerasinghe, and K. Gamage. 2006. Sinhala grapheme-to-phoneme conversion and rules for schwa epenthesis. In ACL. find
  31. Robert Weide. 2005. The carnegie mellon pronouncing dictionary [cmudict. 0.6]. Pittsburgh, PA: Carnegie Mellon University. find
  32. J. Wells. 1995. Computer-coding the ipa: a proposed extension of sampa. find
  33. Chiranjeevi Yarra, Ritu Aggarwal, Avni Rajpal, and Prasanta Kumar Ghosh. 2019. Indic timit and indic english lexicon: A speech database of indian speakers using timit stimuli and a lexicon from their mispronunciations. In 2019 22nd Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), pages 1–6. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jain-2021-ie-cps-lexicon-automatic-speech/