📄 Learning Bilingual Lexicons from Monolingual Corpora

Summary

The paper addresses how to build a word-translation dictionary when no parallel text exists for a language pair, using only two separate monolingual corpora and, optionally, a few seed translations. Each word type is represented by monolingual features: counts of surrounding context words and character substrings. The authors propose matching canonical correlation analysis (MCCA), a generative model in which translation pairs share a latent, language-independent concept vector projected into each language's feature space. Words without a translation are generated from a separate baseline distribution. Learning uses a hard-EM loop. The E-step finds the best bipartite matching with the Hungarian algorithm, and the M-step fits probabilistic CCA to the current pairs. The number of accepted pairs grows gradually over iterations. Experiments cover English paired with Spanish, French, Chinese, Arabic and German, using Wikipedia, Europarl, Gigaword, Xinhua and UN data. On topically similar but non-parallel English-Spanish text, the best setup reaches 89.0% precision at 33% recall. This shows that useful lexicons can be mined for resource-poor pairs.

Key points

• Corpora. EN-ES-W uses 3,851 Wikipedia articles that exist in both languages. EN-ES(FR)-D, EN-CH-D and EN-AR-D take the first 50k sentences of a parallel corpus for English and the second 50k for the other language, so no sentence is shared. EN-ES-G uses 100k sentences each of English and Spanish Gigaword. Alignments are never used.

• Evaluation setup. Only nouns are scored, following Koehn and Knight (2002). A word counts as a noun if its most frequent tag is a noun.

• Languages without useful spelling overlap. With context features alone, English-Chinese and English-Arabic reach 26.8 and 31.0 precision at 0.33 recall. At 0.1 recall they reach 60.1 and 70.0.

• Comparison with prior work. In a setup approximating Koehn and Knight (English Gigaword vs. German Europarl), MCCA scores 61.7% accuracy on its 186 most confident outputs. Koehn and Knight reported 39%.

• Error analysis on EN-ES-W. Of the top 100 "errors":

  • 21 were valid translations missing from Wiktionary;
  • 30 were semantically related words;
  • 15 were orthographic look-alikes;
  • 4 were morphological variants;
  • 30 were uncategorized.

  Adding an alignment-derived lexicon to the reference raised precision at 0.33 recall from 92.3 to 98.0.

Interesting ideas

• I1. Alternate between matching and projection fitting. The loop switches between solving a one-to-one assignment problem and refitting a shared projection space. This hard-EM pattern could carry over to other unsupervised alignment tasks, such as linking entities, place names or product categories across two languages.
• I2. Grow the trusted set slowly. Only the highest-scoring pairs are kept at first, and the number of retained edges rises with each iteration. This keeps early noise from corrupting the model and is a cheap safeguard for any self-training loop.
• I3. Use a cheap proxy score. The exact likelihood-based edge weight was replaced by a constant minus the squared distance between the words' expected latent vectors. This was faster and even slightly more accurate, a reminder that a simple geometric score can beat a principled but costly one.
• I4. Simulate a non-parallel setting from parallel data. Different halves of a parallel corpus serve as the two monolingual corpora. This gives realistic comparable data while the domain stays controlled, which helps when building benchmarks for low-resource pairs.
• I5. Audit the reference dictionary itself. Manually categorizing the top errors revealed many correct pairs absent from Wiktionary, and a supplementary lexicon changed the measured precision considerably. Projecting features into the shared space also gave interpretable correspondences, such as suffix mappings. Both checks are worth repeating whenever a gold lexicon may be incomplete.