Elamur Research⌕
☰
home / papers / 1990 / A Statistical Approach to Machine Translation

A Statistical Approach to Machine Translation

Paper · 1990 · Peter F. Brown, John Cocke, Stephen A. Della Pietra et al.

Summary

The authors from IBM return to an old idea of Warren Weaver: machine translation can be done statistically rather than with hand-written rules. They formulate the task as a search for the source sentence that most probably gave rise to the observed sentence. Bayes' theorem decomposes this task into three parts. The first is a source language model, here an n-gram model. The second is a translation model, in which each word has a fertility probability, i.e., how many words it generates, as well as lexical probabilities and probabilities of word-order distortion. The third is a decoder that searches for the best sentence using an inexact, suboptimal search. The parameters of both models are estimated automatically from the Canadian Hansard parallel corpus. The approach was tested on French-to-English translation in a task with a vocabulary of approximately 1,000 words. Only a small fraction of sentences matched the human translation exactly. Nevertheless, almost half of the outputs can be considered acceptable translations, and editing the system's output required noticeably fewer keystrokes than translating from scratch. This work laid the foundation for statistical machine translation (the IBM models), on which the systems of the following two decades relied.

Key points

  • Noisy channel scheme. One seeks the S for which the product Pr(S)·Pr(T|S) is largest. The language model is responsible for word order, and the translation model for word choice.
  • The "bag translation" experiment. The words of an English sentence were shuffled, and their order was then restored with a trigram model. Of 38 sentences shorter than 11 words, 24 (63%) were restored exactly, and another 8 preserved the meaning, 84% in total.
  • Translation model parameters. Fertility up to 25, lexical probabilities Pr(f|e), and a very simple distortion model Pr(i|j,l), where positions and lengths are limited to 25. Words without a counterpart are generated by a special word (null).
  • Test on 73 sentences from the Hansard. Exact translations accounted for 5%, another 25% were synonymous phrasings, and 18% were different correct translations, so acceptable outputs totaled 48%. Ungrammatical outputs accounted for 37%. Correction took 776 keystrokes, whereas translation from scratch took 1,916, i.e., approximately 60% less work.
  • Limitations and plans. The models were trained on only ~1% of the available data for translation and ~10% for the language model. The perplexity of the bigram model was ~78, while ~9 was expected for the trigram model. The plans include morphology, multi-word units such as "to go → aller", and syntactic grammars.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬IBM Research - Thomas J. Watson Research CenterYorktown Heights · United States55 works · 49,277 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 1,105 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Natural language processing · 131 works

Cite

Cite as Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer and Paul S. Roossin. 1990. A Statistical Approach to Machine Translation. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@misc{brown1990statistical,
  title = {{A Statistical Approach to Machine Translation}},
  author = {Peter F. Brown, John Cocke, Stephen A. {Della Pietra}, Vincent J. {Della Pietra}, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer and Paul S. Roossin},
  year = {1990},
  doi = {10.5555/92858.92860},
  url = {https://doi.org/10.5555/92858.92860}
}

References

12 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bahl, L. R.; Jelinek, F.; and Mercer, R. L. 1983 A Maximum Likelihood Approach to Continuous Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-5(2): 179-190. find
  2. Baker, J. K. 1979 Stochastic Modeling for Automatic Speech Understanding. In: Reddy, R. A. (ed.) Speech Recognition. Academic Press, New York, NY. find
  3. Baum, L. E. 1972 An Inequality and Associated Maximization Technique in Statistical Estimation of Probabilistic Functions of a Markov Process. Inequalities 3:1-8. find
  4. Bernstein, R. 1988 Howard's Way. The New York Times Magazine 138(47639): pp 40-44, 74, 92. find
  5. Dempster, A. P.; Laird, N. M.; and Rubin, D. B. 1977 Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society 39(B):1-38. find
  6. Ferguson, J. D. 1980 Hidden Markov Analysis: An Introduction. In: Ferguson, J. D. (ed.), Hidden Markov Models for Speech. IDA-CRD, Princeton, NJ. find
  7. Garside, R. G.; Leech, G. N.; and Sampson, G. R. 1987 The Computational Analysis of English: A Corpus-Based Approach. Longman, NY. find
  8. Jelinck, F. and Mercer, R. L. 1980 Interpolated Estimation of Markov Source Parameters from Sparse Data. In: Proceedings of the Workshop on Pattern Recognition in Practice. North-Holland, Amsterdam, The Netherlands. find
  9. Sampson, G. R. 1986 A Stochastic Approach to Parsing. Proceedings of the I I th International Conference on Computational Linguistics. 151-155. find
  10. Sharman, R. A.; Jelinck, F.; and Mercer, R. L. 1988 Generating a Grammar for Statistical Training. In: Proceedings of the IBM Conference on Natural Language Processing, Thornwood, NY. find
  11. Sinclair, J. M. 1985 Lcxicographic Evidence. In: Ilson, R. (ed.) Dictionaries, Lexicography and Language Learning. Pergamon Press, N e w York, NY. find
  12. Weaver, W. 1955 Translation (1949). In: Machine Translation of Languages, M I T Press, Cambridge, M A . find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/brown-1990-statistical-approach-machine-translation/