📄 Learning a Translation Lexicon from Monolingual Corpora

Summary

Statistical MT usually learns word translations from parallel text. Such text is scarce for most language pairs and often limited to parliamentary or legal domains, while monolingual text is plentiful. Koehn and Knight test how far one can get building a one-to-one German–English noun lexicon from two unrelated monolingual news collections: the 1990–1992 Wall Street Journal and the 1995–1996 German DPA newswire.

They first build a seed of 1,339 pairs. These are words spelled the same in both languages, plus words that match after a few simple letter-rewriting rules. They then test four further clues: spelling similarity, shared context, preserved similarity between words, and frequency. Each clue scores candidate pairs, and a greedy one-to-one matching over 1,000 frequent nouns picks the final entries.

Spelling and context each add over a hundred correct entries, and summing their scores works best. Accuracy against a parallel test corpus rises from 15.8% with the seed alone to about 39%. The work matters because it builds a usable bootstrap lexicon with no bilingual data, which helps low-resource languages and new domains.

Key points

• Seed lexicon from identical spellings
  • 976 German nouns have an English word with exactly the same spelling, and 88% of these are real translations.
  • Reliability grows with word length: about 60% for 3-letter words versus 98% for 10-letter words.
  • Keeping only words of 6 or more letters gives 622 pairs at 96% accuracy.
• Rewriting rules
  • Two German-to-English rules (k/z → c, and -tät → -ty) add 363 pairs, of which 330 (91%) are correct.
  • Together with the identical words, this gives the 1,339-pair seed.
• Spelling and context clues
  • Spelling similarity uses the longest common subsequence ratio (Melamed, 1995).
  • Context vectors follow Rapp (1999), but use a four-noun window that skips all non-noun words in between.
  • The context vectors are translated through the seed pairs so they can be compared across languages.
• Results on 1,000 frequent nouns
  • Spelling alone adds 140 correct entries (25.4% corpus score), and context alone adds 107 (31.9%).
  • The similarity and frequency clues add only 2 entries each.
  • Spelling+context gives 185 entries (38.6%), and all clues together give 186 (39.0%).
  • A random baseline gets about 1 correct entry, while a lexicon learned from a large parallel corpus reaches 89%.
• Cost
  • The greedy matching is quadratic in vocabulary size and took up to 3 days for 1,000 words.
  • The authors estimate a run on 10,000 words would take nearly a year.

Interesting ideas

• I1. Free seed dictionary from shared strings. Identically spelled words, filtered by length, give a cheap and fairly precise seed. This is attractive for any language pair that shares loanwords, international terms or names; in a Cyrillic–Latin setting it would first need transliteration.
• I2. Breaking the chicken-and-egg loop. Context vectors normally need a dictionary to translate them. Here the seed pairs serve as the "translatable" context dimensions, and this bootstrapping pattern can be reused for other cross-lingual mappings.
• I3. Flooring weak similarity scores. All spelling scores below 0.3 are treated as 0.3, because such small overlaps are accidental. This stops noise from dominating when several signals are added, a trick worth applying in other score-fusion setups.
• I4. Greedy one-to-one assignment. The highest-scoring pair is fixed, both words are removed, and the process repeats. This is a simple, practical alternative to exhaustive matching, though it is still costly at large vocabulary sizes.
• I5. Two complementary evaluation views. The paper counts correct dictionary entries and also measures token-weighted accuracy against a parallel corpus. This shows which clue helps on frequent words (context) versus on many rarer types (spelling).