📄 Transliteration Experiments on Chinese and Arabic

Summary

The paper presents the system the University of Alberta submitted for the machine transliteration competition held at NEWS 2012. It covers two directions: English to Chinese (EnCh) and Arabic to English (ArEn). The base system is DirecTL+, a discriminative online learner with joint n-gram features. It is trained on many-to-many alignments produced by M2M-Aligner.

For Chinese, the authors convert the target characters to Pinyin, but only to help the alignment step; the aligned pairs are mapped back to characters before training. They compare two Pinyin encodings, plain letters and syllable units (initials and finals), and also merge the two models by simple voting.

For Arabic, they test a hand-made mapping that collapses letters sharing one English equivalent. The syllable-based Pinyin gave a small gain, and the combination a marginal further one. The Arabic letter mapping slightly reduced accuracy. The final systems placed second (EnCh) and third (ArEn) by top-1 accuracy, each within 0.5% of the winner. Both had the best F-score among primary runs.

Key points

• Chinese characters in the task data were reduced from about 500 distinct symbols to 26 Pinyin letters, or to 47 initials/finals (PINYIN-SYL), for M2M alignment. About 50 characters with several readings were resolved by hand toward their usual name pronunciation.
• On the cleaned EnCh development set (2483 entries, after splitting roughly 3% multi-word entries), the systems scored as follows:
  • PINYIN-LET: top-1 0.296, F-score 0.679
  • PINYIN-SYL: top-1 0.302, F-score 0.681
  • COMBINED: top-1 0.304, F-score 0.682
• On the official EnCh test, COMBINED (the primary run) reached top-1 0.325 and F-score 0.672. PINYIN-SYL reached 0.325 and 0.673.
• For ArEn development, NO-MAP scored top-1 0.529 versus 0.519 for LETTER-MAP. On the test set, NO-MAP reached top-1 0.583 and F-score 0.933. This was consistent with earlier findings that hand-crafted transliteration rules rarely help.
• Two other ideas failed:
  • Generating in Pinyin and decoding back to characters with a trigram language model gave poor accuracy.
  • A third model trained directly on raw characters weakened the combination.

Interesting ideas

• I1. Use a simpler intermediate representation only during alignment, then project the alignment back to the original symbols, so no lossy reverse conversion is ever needed at output time.
• I2. Choose alignment units that match real phonological structure (syllable initials and finals) rather than raw letters, because multi-letter units often do not sound like their parts.
• I3. In a voting combination, rank systems by development accuracy and use that rank to break ties; also note that near-identical models gain little from combination.
• I4. Treat a plausible hand-made normalization as a hypothesis and test it against an unmodified baseline; here, collapsing symbols removed useful information without reducing ambiguity enough.
• I5. Check development data for inconsistencies with the training and test sets, such as multi-word entries that exist only in dev, and clean them before tuning.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): when aligning silver data from different witness parsers or tokenizations, align on a normalized form (e.g., lemmas or lowercase) and project links back to surface tokens, avoiding ambiguous de-normalization.
• I2 (tokens): for morphology and lemma alignment, segment words into linguistically real units (stems, suffixes like -ing/-ed) instead of characters to get cleaner learned correspondences.
• I3 (learn): when merging witness parsers for silver trees, weight ties by each parser's measured accuracy, and prefer witnesses that differ in principle rather than near-clones.
• I4 (learn): every self-induced repair or normalization rule should be checked against a no-rule control on held-out data before adoption, discarding rules that lose information.
• I5 (answer): audit evaluation sets for formatting quirks absent from real inputs, so test scores reflect true reader ability rather than artifacts.