📄 Epitran: Precision G2P for Many Languages

Summary

Many speech and NLP pipelines need a grapheme-to-phoneme (G2P) step. Research has mostly targeted hard orthographies such as English, while many low-resource languages with fairly transparent spelling lack good tools. Existing options fall short: Unitran handles no Roman scripts and cannot tell apart languages sharing a script, such as Amharic and Tigrinya. URoman only romanizes and does not produce phonemes. The authors present Epitran, an open-source Python library (MIT license) that turns orthographic word tokens into IPA or X-SAMPA for 61 languages. It sits on top of several back-ends. Flite serves English, Epihan serves Chinese, and a rule-based "Simple Epitran" covers everything else, using CSV mapping tables plus optional pre- and post-processing rewrite rules. Because the output is rule-based, every word gets a pronunciation and the mapping is fully consistent. In ASR experiments, Epitran-based pronunciations lowered word error rate compared with Babel baselines. The tool also supported cross-lingual transfer, entity linking and annotation work in DARPA LORELEI. This makes it a practical way to place many languages in one shared phonetic space.

Key points

• Epitran ships with 61 languages and outputs IPA, which works with PanPhon, or X-SAMPA. It runs on Python 2 and 3, and its main API methods are transliterate, xsampa_list, trans_list and trans_delimiter.

• It has three back-ends:

  • Flite (lex_lookup), which is used to handle English.
  • Epihan, which covers Chinese in both its Simplified and Traditional scripts and requires the CC-CEDict dictionary.
  • Simple Epitran for all other languages, which needs no extra software.

• Modes are defined per language-script pair (e.g. tir-Ethi, kaz-Cyrl, aze-Latn). Variants include:

  • naive phonemic (-np);
  • reduced phoneme inventory (-red);
  • preprocessed (-pp);
  • Babel-oriented (-bab).

  Arabic, Persian and Urdu are rendered without short vowels.

• For ASR, the authors trained models on six IARPA Babel releases, and models using Epitran lowered WER compared with the Babel baselines.

• Other uses within the ARIEL/LORELEI team:

  • transferring ASR models from Amharic to Tigrinya and Oromo;
  • merging Amharic datasets from different sources;
  • transferring NER from Uzbek and Turkish to Uyghur (Bharadwaj et al., 2016);
  • training a polyglot MT model on IPA;
  • entity linking into English.

Interesting ideas

• I1. Two-column CSV maps with greedy longest-match. The order of rows does not matter, so tables copied from Wikipedia, Omniglot or grammar books can be used almost directly. This keeps the barrier low for adding a new language or dialect.
• I2. Rewrite-rule cascades before and after the mapping. These rules support named constants, such as a regex class for consonants. The Tigrinya example inserts a predictable vowel this way, showing that a simple table plus a few context rules can cover non-trivial orthographies.
• I3. Plain Python and regex instead of XFST/Foma. The authors kept dependencies limited to pip, avoided reordering rules programmatically, and made metathesis easy to write. This is a good reminder that the theoretically elegant tool is not always the most maintainable one.
• I4. Modes keyed by language plus script, with alternates. Separate modes for the same language (reduced inventory, naive phonemic) let users trade precision against a smaller label set. This design suits any multi-script or multi-variety resource, and it makes explicit choices such as which Spanish variety to model.
• I5. Rule-based G2P as a consistency and readability tool. Deterministic output avoids the label noise that comes from annotators disagreeing on similar phonemes. IPA output also lets non-speakers annotate named entities in unfamiliar scripts, which is a cheap help for annotation in low-resource projects.