📄 Is word-to-phone mapping better than phone-phone mapping for handling English words?

Summary

The paper looks at how a Telugu text-to-speech system should pronounce English words that appear inside Telugu text. Such code mixing is common in Indian web writing; in one Telugu blog, English made up roughly 20–30% of the text. The authors use only the native Telugu phone set and compare two options. The first swaps each US English phone for the closest Telugu phone (phone-phone). The second maps the whole English spelling directly to a Telugu phone sequence (word-phone). In a small manual pilot, native listeners clearly preferred word-phone mapping. The authors then learned these mappings automatically. They aligned letters to phones with an EM-style epsilon scattering procedure and trained CART letter-to-sound models. Word-level accuracy for English-word-to-Telugu-phone mapping was higher than for the CMU dictionary English-phone mapping, likely because Telugu speakers tend to pronounce words as they are spelled. Once the learned rules were added to an HMM-based Telugu TTS system, listeners rated it almost the same as hand-written mappings. Bilingual TTS can therefore handle foreign words without a combined phoneset or a multilingual speaker.

Key points

• Pilot study: 10 bilingual utterances were rated by 5 native listeners. Word-phone mapping scored MOS 3.48 and phone-phone mapping 2.66. In ABX, listeners chose word-phone 32/50 times, phone-phone 4/50, and no preference 14/50.
• Alignment and models: letters were aligned to phones with automatic epsilon scattering (Pagel et al., 1998), which places "empty" phones for silent letters. Two CART models were trained: EW-EP, based on the CMU pronunciation dictionary, and EW-TP, mapping English words to Telugu phones.
• Test setup: the held-out set was every ninth lexicon word, plus a separate test set from another domain. Accuracy was measured per letter and per word, and training sizes ranged from 1,000 to 100,000 words.
• Accuracy results: EW-TP trained on 10,000 words reached 71.67% word accuracy on held-out data and 70.9% on the other domain, with about 95–97% letter accuracy. EW-EP stayed near 55–56% word accuracy even with 100,000 training words.
• Final listening test: 25 sentences were judged by 10 listeners. The manual system (TM) scored MOS 3.48 and the automatic system (TA) 3.43. In ABX, listeners chose TM 51/250 times, TA 38/250, and no preference 161/250.

Interesting ideas

• I1. An EM loop alternates between estimating letter–phone match probabilities and re-aligning each word, inserting empty slots where a letter has no phone; this turns uneven sequence pairs into fixed-length training examples for a simple classifier.
• I2. Before automating, the authors hand-built a few outputs for both competing designs and had listeners compare them, settling the design choice cheaply before any engineering.
• I3. Accuracy is reported against training-set size and on both a held-out split and an out-of-domain set, which shows whether gains come from data volume or fail to transfer.
• I4. The target representation is chosen to fit the listener's inventory (native phones) rather than to faithfully copy the source language's units, and listeners preferred this.
• I5. The ABX test includes an explicit "no preference" option, so a near-tie between the manual and automatic systems shows up as evidence of parity rather than forced noise.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): the same alignment could pair word forms with lemmas or morphological analyses, so that inflection or lemmatization edit rules can be learned per character and remain explainable.
• I2 (learn): when choosing between two representations (e.g., two frame encodings), hand-annotate a handful of sentences both ways and run a quick preference check before building either pipeline.
• I3 (learn): parser and tagger evaluations could routinely include learning curves and an out-of-domain test set, such as fables versus school texts, to detect overfitting to silver data.
• I4 (back): generation and translation through the meta-world could render concepts in the reader's own vocabulary and constructions, instead of mirroring the source structure unit by unit.
• I5 (learn): when LLM teacher tiers or human judges compare two parser or repair-rule versions, allowing a "no difference" vote would separate real improvements from cosmetic changes.