Elamur Research⌕
☰
home / papers / 2006 / N-gram-based Machine Translation

N-gram-based Machine Translation

Summary

Mariño and colleagues present a statistical MT system in which translation is modelled as a language model over a bilingual token stream. Each sentence pair is cut into minimal bilingual chunks called tuples, and a back-off n-gram model over these tuples serves as the translation model. Tuples come from many-to-many word alignments, taken as the union of the two IBM alignment directions. The segmentation must be monotonic and unique, and no smaller split may break an alignment link. The tuple model is combined log-linearly with four additional feature functions, including a target language model, a word bonus and IBM-based lexicon features. Decoding uses the MARIE decoder in monotone mode, with no reordering. The system was tested on Spanish↔English European Parliament Plenary Sessions (EPPS) data, on text, verbatim transcripts and ASR output. It matched or beat leading systems in the TC-STAR evaluation and the ACL 2005 shared task. This matters because it showed that a finite-state-inspired bilingual n-gram model can compete with phrase-based SMT for closely related languages.

Key points

  • The translation model approximates the joint probability p(T, S) as a product of tuple n-gram probabilities, so the conditioning context is bilingual.
  • Tuples are extracted from the union of source-to-target and target-to-source IBM alignments. The authors found the union worked best and attribute this to its higher recall.
  • In the TC-STAR evaluation (EPPS), the n-gram system ranked first on final text editions in both directions (ES→EN 53.3 BLEU, EN→ES 46.2 BLEU). It ranked second or third on verbatim transcripts and ASR output.
  • In the ACL 2005 shared task, it placed second for FR→EN (30.20), ES→EN (30.07) and DE→EN (24.26), and third for FI→EN (20.31).
  • Spanish→English output was consistently better than English→Spanish. The authors point to morphology as the main issue in the harder direction, and they also flag reordering, null-aligned words and verb forms as recurring error sources.

Interesting ideas

  • I1. Translation as a language model over a "bilanguage". Merging source and target into joint units lets standard back-off n-gram tooling handle smoothing and context for free. The same trick could apply to other paired-sequence tasks, such as transliteration or normalisation.
  • I2. Deterministic minimal segmentation. Defining units as the smallest chunks that respect alignment links and keep monotonic order produces a unique, compact vocabulary with no maximum-length parameter. This is far smaller than an exhaustive phrase table.
  • I3. Patching coverage gaps with one-word fallback units. Some words only ever appear inside larger tuples ("embedded words"), so the authors add single-word tuples from a bilingual dictionary. This guarantees every seen word has a translation option, a cheap fix for any chunk-based model.
  • I4. Controlled comparison through shared components. The n-gram system was compared with a phrase-based system that uses the same decoder, language model and features. Isolating the core modelling choice this way is a sound evaluation design worth copying.
  • I5. Knowing when the method fits. The authors note that tuple length grows with structural divergence between languages, so the approach suits close pairs and suffers from sparsity on distant ones. This is a useful check before applying it to typologically different pairs, such as Ukrainian with a non-Slavic language.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

84 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 83 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.2006.32.4.527

@article{marino2006n,
  title = {{N-gram-based Machine Translation}},
  author = {Mariño, José and Banchs, Rafael E. and Crego, Josep M. and Gispert, Adrià de and Lambert, Patrik and Fonollosa, José A. R. and Costa-jussà, Marta R.},
  journal = {Computational Linguistics},
  year = {2006},
  doi = {10.1162/coli.2006.32.4.527},
  url = {https://doi.org/10.1162/coli.2006.32.4.527}
}

References

37 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Banchs, Rachel E., Josep Maria Crego, Adrià de Gispert, Patrik Lambert, and José Bernardo Mariño. 2005. Statistical machine translation of Euparl data by using bilingual n-grams. In ACL Workshop on Data-Driven Machine Translation and Beyond, pages 133–136, Ann Arbor, MI. find
  2. Bangalore, Srinivas and Giuseppe Riccardi. 2000. Stochastic finite-state models for spoken language machine translation. find
  3. In Proceedings of the Workshop on Embedded Machine Translation Systems, pages 52–59, Seattle, WA. find
  4. Berger, Adam, Stephen Della Pietra, and Vincent Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. find our card
  5. Brown, Peter, John Cocke, Stephen Della Pietra, Vincent Della Pietra, Frederick Jelinek, John Lafferty, Robert Mercer, and Paul S. Roossin. 1990. A statistical approach to machine translation. Computational Linguistics, 16(2):79–85. find
  6. Brown, Peter, Stephen Della Pietra, Vincent Della Pietra, and Robert Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. find
  7. Casacuberta, Francisco. 2001. Finite-state transducers for speech input translation. In Proceedings IEEE ASRU, pages 375–380, Madonna di Campiglio, Italy. find
  8. Casacuberta, Francisco and Enrique Vidal. 2004. Machine translation with inferred stochastic finite-state transducers. Computational Linguistics, 30(2):205–225. find
  9. Costa-jussà, Marta Ruiz, José Adrián Rodriguez Fonollosa, and Enric Monte. 2006. Using reordering in statistical machine translation based on alignment block classification. Internal Report. http://gps-tsc.upc.es/veu/personal/mruiz/docs/br06.pdf. ↗
  10. Crego, Josep Maria, José Bernardo Mariño, and Adrià de Gispert. 2004. Finite-state-based and phrase-based statistical machine translation. In Proceedings of the 8th International Conference on Spoken Language Processing, pages 37–40, Jeju, Korea. find
  11. Crego, Josep Maria, José Bernardo Mariño, and Adrià de Gispert. 2005a. An Ngram-based statistical machine translation decoder. In INTERSPEECH 2005, pages 3185–3188, Lisbon, Portugal. find
  12. Crego, Josep Maria, José Bernardo Mariño, and Adrià de Gispert. 2005b. Reordered search and tuple unfolding for Ngram-based SMT. Proceedings of the Tenth Machine Translation Summit, pages 283–289, Phuket, Thailand. find
  13. Crego, Josep Maria, Marta Ruiz Costa-jussà, José Bernardo Mariño, and José Adrián Rodriguez Fonollosa. 2005c. Ngram-based versus phrase-based statistical machine translation. In Proceedings of the International Workshop on Spoken Language Translation, pages 177–184, Pittsburgh, PA. find
  14. Crego, Josep Maria and José Bernardo Mariño. 2006. Integration of POStag-based source reordering into SMT decoding by an extended search graph. In Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas, Boston, MA. de Gispert, Adrià and José Bernard… find
  15. Hutchins, John. 1986. Machine Translation: Past, Present and Future. Ellis Horwood, Chichester, England. find
  16. Kay, Martin, Jean Mark Gawron, and Peter Norvig. 1992. Verbmobil: A Translation System for Face-to-Face Dialog. CSLI. find
  17. Kneser, Reinhard and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 49–52, Detroit, MI. find
  18. Knight, Kevin and Yaser Al-Onaizan. 1998. Translation with finite-state devices. In AI Lecture Notes in Artificial Intelligence, volume 1529, Springer-Verlag, pages 421–437. find
  19. Koehn, Philippe, Franz Joseph Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Meeting of the North American chapter of the ACL, pages 48–54, Edmonton, Alberta, Canada. find
  20. Koehn, Philippe. 2002. Europarl: A multilingual corpus for evaluation of machine translation. Available online at: http://people.csail.mit.edu/people/koehn/publications/europarl/. ↗
  21. Mariño, José Bernardo, Rafael E. Banchs, Josep Maria Crego, Adrià de Gispert, Patrik Lambert, José Adrián Rodriguez Fonollosa, and Marta Ruiz. 2005. Bilingual N-gram statistical machine translation. In Proceedings of the Tenth Machine Translation Summit, pages 275–282, Phuket, Thailand. find
  22. Ney, Hermann, Volker Steinbiss, Richard Zens, Evgeny Matusov, Jorge González, Young-suk Lee, Salim Roukos, Marcello Federico, Muntsin Kolss, and Rafael Banchs. 2005. SLT progress report. TC-STAR Deliverable D5, European Community project no. FP6-506738. Available online at: http://www. tc-star.org/… find
  23. Och, Franz Joseph and Hermann Ney. 2000. Improved statistical alignment models. In Proceedings of the 38th Annual Meeting of the ACL, pages 440–447, Hong Kong, China. find our card
  24. Och, Franz Joseph and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th Annual Meeting of the ACL, pages 295–302, Philadelphia, PA. find our card
  25. Och, Franz Joseph and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  26. Och, Franz Joseph, Daniel Gildea, Sanjeev Khudanpur, Anoop Sarkar, Kenji Yamada, Alexander Fraser, Shankar Kumar, Libin Shen, David Smith, Katharine Eng, Viren Jain, Zhen Jin, and Dragomir Radev. 2004. A smorgasbord of features for statistical machine translation. In Proceedings of the Human Languag… find our card
  27. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Conference of the ACL, pages 311–318, Philadelphia, PA. find our card
  28. Press, William H., Saul Teukolsky, William Vetterling, and Brian P. Flannery. 2002. Numerical Recipes in C++: The Art of Scientific Computing, Cambridge University Press. find
  29. Riccardi, Giuseppe, Roberto Pieraccini, and Enrico Bocchieri. 1996. Stochastic automata for language modeling. Computer Speech and Language, 10(4):265–293. find
  30. Shannon, Claude E. 1949. Communication theory of secrecy systems. Bell System Technical Journal, 28:656–715. find
  31. Shannon, Claude E. 1951. Prediction and entropy of printed English. Bell System Technical Journal, 30:50–64. find
  32. Shannon, Claude E. and Warren Weaver. 1949. The Mathematical Theory of Communication, University of Illinois Press, Urbana, IL. find
  33. Stolcke, Andreas 2002. SRLIM: An extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, pages 901–904, Denver, CO. find
  34. Tillmann, Christoph and Fei Xia. 2003. A phrase-based unigram model for statistical machine translation. In Proceedings of HLT-NAACL - Short Papers, pages 106–108, Edmonton, Alberta, Canada. find
  35. Vidal, Enrique. 1997. Finite-state speech-to-speech translation. In Proceedings of 1997 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 111–114, Munich, Germany. find
  36. Weaver, Warren. 1955. Translation. In William Locke and A. Donald Booth, editors, Machine Translation of Languages: Fourteen Essays. John Wiley & Sons, New York, pages 15–23. find
  37. Zens, Richard, Franz Joseph Och, and Hermann Ney. 2002. Phrase-based statistical machine translation. In 25th German Conference on Artificial Intelligence, pages 18–32, September. Aachen, Springer Verlag. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marino-2006-n-gram-based-machine-translation/