Elamur Research
☰
home / papers / 2005 / Improving Machine Translation Performance by Exploiting Non-Parallel Corpora

Improving Machine Translation Performance by Exploiting Non-Parallel Corpora

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J05-4003.pdf

Summary

Statistical MT needs parallel text, but such data is scarce and mostly comes from parliamentary or UN proceedings. Systems trained on it translate other genres, such as news, poorly. Munteanu and Marcu mine comparable, non-parallel corpora, such as multilingual news feeds, for sentence pairs that are actual translations.

Their pipeline works in three steps:

  • It retrieves candidate English articles for each foreign article.
  • It discards sentence pairs with too little word overlap.
  • It applies a maximum entropy classifier that judges each remaining pair on its own, without relying on document context.

The system needs only a bilingual dictionary and a small parallel corpus. The authors apply it to large Arabic, Chinese and English news collections from the Linguistic Data Consortium. Adding the extracted sentences to UN-trained Arabic-English and Chinese-English systems improves translation of news. The gain is about as large as adding human-translated news data of similar size. With bootstrapping, a usable system can be grown from only 100,000 words of parallel text. This makes the method especially valuable for low-resource language pairs.

Key points

  • Document matching: uses the Lemur IR toolkit with TF-IDF retrieval. Queries are built from the top five dictionary translations of each foreign word. The 20 best English articles are kept, searching only within a five-day window around the foreign article's publication date. Doubling the window to 10 days changed nothing, but removing it hurt performance.
  • Candidate filter: keeps a pair only if the sentence length ratio is at most 2 and at least half the words on each side have a dictionary translation in the other sentence.
  • Low-resource gains: starting from a tiny seed corpus, bootstrapped extraction gave more than 50% improvement over a baseline trained only on the existing parallel data.
  • Efficiency: processing 600k English and 500k Chinese documents took roughly 7 days on 10 processors.
  • Domain dependence: the date window is the only domain-specific component. For other genres it could be swapped for another locality cue, such as clustering technical manuals by product type.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

185 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 178 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/089120105775299168

@article{munteanu2005improving,
  title = {{Improving Machine Translation Performance by Exploiting Non-Parallel Corpora}},
  author = {Munteanu, Dragos Stefan and Marcu, Daniel},
  journal = {Computational Linguistics},
  year = {2005},
  doi = {10.1162/089120105775299168},
  url = {https://doi.org/10.1162/089120105775299168}
}

References

38 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Barzilay, Regina and Noemie Elhadad. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2003), pages 25–32, Sapporo, Japan. find
  2. Brown, Peter F., John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. 1990. A statistical approach to machine translation. Computational Linguistics, 16(2):79–85. find
  3. Brown, Peter F., Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. find
  4. Darroch, J. N. and D. Ratcliff. 1974. Generalized iterative scaling for log-linear models. Annals of Mathematical Statistics, 43:95–144. find
  5. Davis, Mark W. and Ted E. Dunning. 1995. A TREC evaluation of query translation methods for multi-lingual text retrieval. In Fourth Text Retrieval Conference, pages 483–498, Gaithersburg, MD. find
  6. Diab, Mona and Steve Finch. 2000. A statistical word-level translation model for comparable corpora. In Proceedings of the Conference on Content-Based Multimedia Information Access, Paris, France. find
  7. Diab, Mona and Philip Resnik. 2002. An unsupervised method for word sense tagging using parallel corpora. In Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics, pages 255–262, Philadelphia. find
  8. Echihabi, Abdessamad and Daniel Marcu. 2003. A noisy-channel approach to question answering. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 16–23, Sapporo, Japan. find
  9. Fung, Pascale and Percy Cheung. 2004. Mining very non-parallel corpora: Parallel sentence and lexicon extraction vie bootstrapping and EM. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), pages 57–63, Barcelona, Spain. find
  10. Fung, Pascale and Kenneth Ward Church. 1994. Kvec: A new approach for aligning parallel texts. In Proceedings of the 15th International Conference on Computational Linguistics (COLING), pages 1096–1102, Kyoto. find
  11. Fung, Pascale and Lo Yuen Yee. 1998. An IR approach for translating new words from nonparallel, comparable texts. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics, pages 414–420, Montreal. find
  12. Gale, William A. and Kenneth Ward Church. 1991. A program for aligning sentences in bilingual corpora. In Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics, pages 177–184, Berkeley, CA. find our card
  13. Gaussier, Eric, Jean-Michel Renders, Irina Matveeva, Cyril Goutte, and Herve Dejean. 2004. A geometric view on bilingual lexicon extraction from comparable corpora. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 527–534, Barcelona, Spain. find
  14. Koehn, Philipp. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 388–395, Barcelona, Spain. find
  15. Koehn, Philipp and Kevin Knight. 2000. Estimating word translation probabilities from unrelated monolingual corpora using the EM algorithm. In Proceedings of the National Conference on Artificial Intelligence, pages 711–715, Austin, TX. find
  16. Malouf, Robert. 2002. A comparison of algorithms for maximum entropy parameter estimation. In Sixth Conference on Natural Language Learning, Taipei, Taiwan. find
  17. Melamed, Dan I. 1997. A portable algorithm for mapping bitext correspondence. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pages 305–312, Madrid, Spain. find
  18. Melamed, Dan I. 1999. Bitext maps and alignment via pattern recognition. Computational Linguistics, 25(1):107–130. find
  19. Minton, Steven N., Sorinel I. Ticrea, and Jennifer Beach. 2003. Trainability: Developing a responsive learning system. In IJCAI Workshop on Information Integration on the Web, pages 27–32, Acapulco, Mexico. find
  20. Moore, Robert C. 2002. Fast and accurate sentence alignment of bilingual corpora. In Proceedings of the 5th Conference of the Association for Machine Translation in the Americas, pages 135–144, Tiburon, CA. find
  21. Moore, Robert C. 2004. Improving IBM word-alignment model 1. In 42nd Annual Meeting of the Association for Computational Linguistics, pages 519–526, Barcelona, Spain. find
  22. Munteanu, Dragos Stefan, Alexander Fraser, and Daniel Marcu. 2004. Improved machine translation performance via parallel sentence extraction from comparable corpora. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association For Computational Linguist… find
  23. Oard, Douglas W. 1997. Cross-language text retrieval research in the USA. In Third DELOS Workshop on Cross-Language Information Retrieval, pages 1–10, Zurich, Switzerland. find
  24. Och, Franz Josef. 2003. Minimum error rate training for statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167, Sapporo, Japan. find
  25. Och, Franz Josef and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 295–302, Philadelphia. find
  26. Och, Franz Joseph and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  27. Ogilvie, Paul and Jamie Callan. 2001. Experiments using the Lemur toolkit. In Proceedings of the Tenth Text Retrieval Conference, pages 103–108, Gaithersburg, MD. find
  28. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia. find our card
  29. Rapp, Reinhard. 1999. Automatic identification of word translations from unrelated English and German corpora. In Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, pages 519–526, College Park, MD. find
  30. Resnik, Philip and Noah A. Smith. 2003. The web as a parallel corpus. Computational Linguistics, 29(3):349–380, September. find our card
  31. Robertson, E. and S. Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual ACM SIGIR, pages 232–241, Dublin, Ireland. find
  32. Ticrea, Sorinel I. and Steven Minton. 2003. Inducing web agents: Sample page management. In Proceedings of the International Conference on Information and Knowledge Engineering, pages 399–403, Las Vegas, NV, June. find
  33. Utiyama, Masao and Hitoshi Isahara. 2003. Reliable measures for aligning Japanese-English news articles and sentences. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 72–79, Sapporo, Japan. find
  34. Vogel, Stephan. 2003. Using noisy bilingual data for statistical machine translation. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics, pages 175–178, Budapest, Hungary. find
  35. Wu, Dekai. 1994. Aligning a parallel English-Chinese corpus statistically with lexical criteria. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 80–87, Las Cruces, NM. find
  36. Yarowsky, David and Grace Ngai. 2001. Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora. In Proceedings of the 2nd Meeting of the North American Association for Computational Linguistics, pages 200–207, Pittsburgh, PA. find
  37. Yarowsky, David, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the First International Conference on Human Language Technology Research, pages 161–168, San Diego, CA. find
  38. Zhao, Bing and Stephan Vogel. 2002. Adaptive parallel sentences mining from web bilingual news collection. In 2002 IEEE International Conference on Data Mining, pages 745–748, Maebashi City, Japan. find