📄 Identifying Word Translations in Non-Parallel Texts

Summary

The paper asks whether word translations can be found without parallel corpora, since aligned bilingual text is scarce for most domains and language pairs. Rapp's hypothesis is that words that are translations of each other show similar co-occurrence behaviour, even in texts that were written independently. To test this, he took 100 English stimulus words from the Kent & Rosanoff (1910) association list and their German equivalents by Russell (1970). He then built co-occurrence matrices from two unrelated corpora: 33 million words of English and 46 million words of German. After normalising both matrices, he scrambled the German word order at random. He compared the scrambled German matrix to the English one using a sum of absolute cell differences. Similarity was best when the word orders matched and worsened steadily as more positions were wrong. This shows that cross-lingual co-occurrence patterns correlate strongly even across non-parallel text. The study is an early proof of concept for extracting bilingual lexicons from comparable corpora.

Key points

• Data. The English corpus combines the Brown Corpus, Wall Street Journal texts, Grolier's Electronic Encyclopedia and scientific abstracts. The German corpus is mostly newspaper text from Frankfurter Rundschau, Die Zeit and Mannheimer Morgen. The author believes the two corpora share no parallel passages.
• Co-occurrence window. Two words count as co-occurring if no more than 11 other words lie between them.
• Association measures. Three measures were compared:
  • raw counts;
  • a mutual-information-like ratio with the logarithm dropped, because the log is undefined for zero counts;
  • a frequency-dampened score from Wettler & Rapp (1993): squared joint frequency divided by the product of the two word frequencies.
• Normalisation and scale. Each matrix was scaled so that its entries sum to the number of cells. The difference score therefore runs from 0 (identical matrices) to 20 000 (complete mismatch) for the 100×100 matrices.
• Results. For every number of misplaced words, 1000 random permutations were sampled. All three curves rise monotonically, and the Wettler & Rapp score discriminates best. However, the minimum-value curve shows that some heavily wrong orderings still score deceptively well.

Interesting ideas

• I1. Translation as permutation matching. Bilingual lexicon induction can be framed as finding the row/column permutation that best aligns two monolingual co-occurrence matrices. This reframing makes the task usable for any language pair that has decent monolingual corpora, such as Ukrainian paired with a less-resourced language.
• I2. Controlled degradation as evaluation. Start from a known correct mapping, corrupt it step by step, and plot the similarity score against the amount of corruption. This is a cheap way to check whether a signal can discriminate at all before building a full search algorithm. It also lets you compare candidate association measures on the same footing.
• I3. Dampening word frequency. Scoring co-occurrence in a way that reduces the influence of word frequency gave a much sharper cross-lingual signal than raw counts. This is a reminder that frequency normalisation choices can matter more than the matching procedure itself.
• I4. Seed anchors to limit the search. Translations already known in advance can be fixed in place to shrink the permutation search space. Combining this with many local minima helps reveal which correspondences are consistently reliable. The same idea suits any noisy alignment problem that comes with a small trusted seed dictionary.
• I5. Soft rather than binary decisions. Scoring translations with continuous probabilities, instead of accepting or rejecting each one, accommodates ambiguous words. It also allows the two vocabularies to differ in size, which is useful for real lexicon building where one-to-one mappings rarely hold.