🔁 Machine translation

A survey · about 19 minutes to read · every paper named here has its own page in this library

Machine translation (MT) is the use of computer programs to render text or speech in one human language as text or speech in another. It is one of the oldest problems in computing: proposals to mechanise translation date back to the late 1940s, and the first public demonstration of a translating program took place in 1954. For decades the problem served as a proving ground for whole schools of thought about language — hand-written grammars and dictionaries, probabilistic models estimated from bilingual text, and finally neural networks trained end to end. Each wave reshaped not only translation but natural language processing as a whole; the attention mechanism and the Transformer architecture, now at the heart of large language models, were first developed for translation.

This topic gathers the papers in this library on translation systems and on everything that surrounds them: word and sentence alignment, decoding and tuning, the parallel corpora that feed the models, and, prominently, the question of how translation quality should be measured. Evaluation is unusually central here. Automatic metrics such as BLEU, METEOR, chrF and COMET, the annual shared tasks of the Conference on Machine Translation (WMT), and the debates about whether machines have reached human parity make up a large share of the most cited work.

The collection spans five decades, from early rule-based and interlingua research to multilingual neural systems and translation with large language models. Its centre of gravity lies in the statistical and early neural eras, with a renewed surge of papers in the most recent years on low-resource languages, document-level translation and evaluation with learned metrics.

What it is

At its simplest, a translation system maps a source sentence to a target sentence that preserves its meaning and reads naturally in the target language. These two demands — adequacy (faithfulness to the source) and fluency (well-formedness in the target) — have shaped both system design and evaluation since the field began. Real translation also involves register, terminology, cultural reference, document-level coherence and the many cases where languages simply do not line up word for word.

Several core notions recur across all eras:

• Parallel corpora (bitexts): collections of texts paired with their translations, aligned at the document, sentence and often word level. They are the raw material of every data-driven system.
• Alignment: the correspondence between units of the source and the target — sentences, phrases or words. Learning it, explicitly or implicitly, is the heart of translation modelling.
• Reordering: languages differ in word order (English subject–verb–object against Japanese subject–object–verb, German verb-final subordinate clauses), so a system must decide where each piece goes.
• Decoding: searching among the very large number of possible outputs for the one the model scores highest, usually with beam search.
• Evaluation: judging output quality, either by human assessment or by automatic metrics that compare system output with reference translations.

Machine translation differs from its neighbours in its output: unlike classification tasks, it produces open-ended text, and there are always many correct answers. This links it closely to text generation and summarisation, which inherited much of its methodology, and to multilingual & cross-lingual NLP, which shares its concern with many languages but usually aims at analysis rather than generation. Several techniques first developed for translation — phrase-based models, paraphrase extraction from bitexts, sequence-to-sequence learning — were later applied to monolingual rewriting tasks such as paraphrasing, simplification and error correction, and some of those papers stand on this topic too.

History and milestones

Rule-based and interlingua systems

The first generation of MT systems, from the 1950s to the 1980s, relied on bilingual dictionaries and hand-written rules. Direct systems replaced words and applied local fixes; transfer systems parsed the source, transformed its structure into a target structure, and generated output; interlingua systems aimed to map the source into a language-neutral meaning representation from which any target could be produced. The 1966 ALPAC report in the United States judged early progress disappointing and sharply reduced funding, but rule-based systems such as SYSTRAN remained in commercial use for decades. Work from this period in the library includes Bonnie Dorr's formal account of translation divergences — systematic structural mismatches, such as one language expressing with a verb what another expresses with a preposition — and a review of the ARPA MT evaluation methodologies from the early 1990s.

The statistical turn

In the late 1980s a group at IBM proposed treating translation as a problem of probability estimation from bilingual text. Their paper A Statistical Approach to Language Translation introduced the noisy-channel view: a target sentence is chosen by combining a language model, which judges fluency, with a translation model, which judges how well the target accounts for the source. The family of IBM word-alignment models that followed, trained on the Canadian parliamentary Hansards, defined the field for the next decade. Practical prerequisites were solved at the same time, notably the length-based sentence alignment program of Gale and Church.

Around 2000 the unit of translation moved from words to phrases. The alignment template approach of Och, Tillmann and Ney, refinements of the alignment models together with a gold-standard alignment error rate, the joint phrase model of Marcu and Wong, and the log-linear framework that replaced the fixed noisy-channel product with weighted feature functions led to phrase-based statistical MT. The open-source Moses toolkit (2007) made this architecture the standard research baseline. In the same period, the BLEU metric (2002) gave researchers a cheap, repeatable way to measure progress, and annual shared tasks — NIST evaluations and, from 2006, the Workshop on Statistical Machine Translation — turned the field into a sequence of public benchmarks.

Syntax and hierarchy

A parallel line of work sought to bring linguistic structure into statistical models. Yamada and Knight modelled translation as operations on a parse tree; Chiang's hierarchical phrase-based model learned a synchronous grammar with gaps from unannotated bitext; Galley and colleagues scaled up syntactic rule extraction; dependency treelet translation used source dependency parses. Others kept the phrase-based engine but reordered the input with rules, as in clause restructuring for German.

The neural turn

From 2013 neural networks entered translation, first as features inside statistical systems — such as the neural network joint model — and then as complete systems. Recurrent continuous translation models showed that a single network could condition a language model on a source sentence. Encoder–decoder models of 2014 compressed the source into a vector, and the attention mechanism of Bahdanau, Cho and Bengio let the decoder look back at every source position. Within two to three years neural MT overtook phrase-based systems in shared tasks and commercial deployment. The Transformer (2017), built entirely on attention, became the standard architecture; subword segmentation, back-translation of monolingual data and multilingual training followed.

Multilinguality, evaluation and large language models

Since about 2018 the field has moved in several directions at once: massively multilingual models covering a hundred languages or more, unsupervised translation from monolingual text, document-level and context-aware translation, and a reckoning with evaluation, as claims of human parity met careful critique and neural metrics replaced string overlap. Most recently, general-purpose large language models have become competitive translators, raising new questions about prompting, hallucination and how to evaluate systems that may already exceed the references they are compared with.

Main problems and tasks

The core task is sentence-level translation of text, but the field divides into many sub-problems and settings:

• Word and sentence alignment: recovering correspondences in parallel text, both as a training step and as a task in itself.
• Reordering: placing translated material in the right order, especially between structurally distant languages.
• Vocabulary and rare words: handling names, numbers, morphological variants and words never seen in training. In neural systems this led to work on large target vocabularies, copying via alignment pointers, subword units and character-level models.
• Domain adaptation: transferring a system trained on parliamentary or news text to medicine, law, software or conversation; see Koehn and Schroeder's experiments and the survey of adaptation for neural MT.
• Low-resource translation: building systems when parallel data is scarce, through transfer, multilinguality, monolingual data and data augmentation.
• Document-level translation: keeping pronouns, terminology, tense and style consistent across sentences.
• Speech translation and simultaneous translation: translating spoken input, sometimes while the speaker is still talking.
• Multimodal translation: using images alongside text, as in the shared task on multimodal MT.
• Quality estimation: predicting the quality of a translation without a reference, starting from sentence-level estimation.
• Automatic post-editing: correcting the output of a black-box MT system, a WMT task since the mid-2010s.
• Evaluation: designing metrics and human protocols that can rank systems reliably.
• Bias and robustness: detecting and reducing stereotyped choices (for example of grammatical gender), hallucinated content and sensitivity to noisy input.

Methods and schools

Rule-based and example-based MT

Rule-based systems encode linguistic knowledge in dictionaries and transfer grammars written by experts. They give predictable, controllable output but are expensive to build and brittle outside their coverage. Example-based MT, proposed in Japan in the 1980s, translates by analogy with stored examples, an idea that in some ways anticipated phrase-based statistics.

Statistical machine translation

Statistical MT estimates all model components from data. The IBM models and the HMM aligner learn word correspondences with expectation maximisation; alignment by agreement showed that training two directional aligners jointly beats intersecting them afterwards. Phrase-based systems extract phrase pairs consistent with the word alignment and score them with relative frequencies; factored models extended phrases with lemmas, part-of-speech tags and morphology. A log-linear model combines translation, reordering and language-model features, and the weights are tuned on development data — at first with minimum error rate training, later with batch tuning methods able to handle thousands of features. Language models themselves grew to web scale, as in the Google work on very large n-gram models with "stupid backoff". Decoding moved from maximum a posteriori search towards minimum Bayes-risk decoding, which picks the output that minimises the risk, averaged over competing hypotheses, measured by the evaluation metric of interest.

The main debate within the statistical school concerned syntax: whether to model hierarchical structure (Hiero, tree-to-string and string-to-tree systems) or to keep flat phrases and add syntactic information as features or preprocessing. Syntax helped most for distant pairs such as Chinese–English and Japanese–English.

Neural machine translation

Neural MT models the whole conditional distribution of the target given the source with a single network trained end to end. The main families are:

• Recurrent encoder–decoders with attention, the first dominant design, studied in Cho et al.'s analysis of encoder–decoder properties and refined with coverage models that discourage over- and under-translation.
• Convolutional models, such as the convolutional encoder of Gehring et al..
• Transformers, built entirely on self-attention. Work in the library examines how to scale their training, train very deep encoders, and which tricks transfer between architectures.
• Non-autoregressive models, which generate all target tokens in parallel for speed, starting from the Non-Autoregressive Transformer.

Training methods beyond maximum likelihood include minimum risk training, reinforcement learning, and sequence-level knowledge distillation, which compresses a large teacher into a small student. Monolingual data enters through language-model fusion, back-translation and its iterative and tagged variants. Decoding research covers beam search strategies, lexically constrained decoding and terminology constraints.

Multilingual, unsupervised and LLM-based translation

Multilingual systems share parameters across many language pairs. Early designs shared a single attention module across language-specific encoders and decoders; later ones used one model for all pairs, enabling zero-shot translation between pairs never seen together and, at Google, a single model for over a hundred languages. Unsupervised MT learns from monolingual corpora only, with denoising and back-translation in a shared latent space or with statistical methods. The newest school uses general large language models as translators, studied in work on prompting, in-context example selection, multilingual evaluation of LLMs and document-level translation with LLMs.

The evaluation debate

A long-running argument concerns what automatic metrics actually measure. String-overlap metrics such as BLEU are cheap and reproducible, but they penalise valid paraphrases and lose their discriminating power as systems improve. Critics have documented inconsistent BLEU reporting, reviewed its validity, and shown that metric rankings can be distorted by outlier systems. The WMT metrics tasks of the 2020s concluded that neural metrics are more reliable, while human evaluation itself moved from crowd-sourced scores towards expert error annotation.

Datasets, resources and benchmarks

Translation research depends on a small number of public data sources and recurring benchmarks:

The WMT findings papers in the library — for 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 and 2024 — document how the benchmark evolved: new language pairs, test sets built only from original source text to avoid translationese, a shift from relative ranking to direct assessment, document context for judges, and test suites that probe specific phenomena.

Open-source toolkits are resources in their own right: OpenNMT, Marian, cdec and, for quality estimation, QuEst.

Landmark papers in this library

BLEU (2002)

Papineni, Roukos, Ward and Zhu proposed scoring a translation by its modified n-gram precision against one or more references, with counts capped by the reference and a penalty for overly short output, all aggregated over a whole test corpus. Its ranking of systems agreed with human judges, and it was fast enough to run after every experiment. BLEU became the default metric for two decades and is by far the most cited paper here; nearly every later metric defines itself against it.

Attention for neural translation (2014)

Bahdanau, Cho and Bengio removed the bottleneck of encoding a sentence into one fixed vector. A bidirectional recurrent encoder produced one annotation per source word, and the decoder learned soft weights over these annotations at every step. Quality on long sentences improved dramatically, and the learned weights resembled word alignments. The attention mechanism introduced here became the basis of the Transformer.

On the Properties of Neural Machine Translation (2014)

Cho and colleagues analysed early encoder–decoder translators and found that their quality fell steeply with sentence length and with the number of unknown words. The diagnosis set the research agenda that attention, large vocabularies and subword units then addressed.

METEOR (2005)

Banerjee and Lavie built a metric on an explicit alignment between hypothesis and reference, matching exact words first and then stems and synonyms, combining precision and a heavier-weighted recall, and penalising fragmented matches. It correlated better with human judgments at the sentence level than BLEU. Successors include Meteor 1.3 and Meteor Universal, which derives its resources for any language from the training bitext.

A Call for Clarity in Reporting BLEU (2018)

Matt Post showed that "BLEU" in published papers hides a family of configurations, chiefly in how references are tokenised, and that these choices alone shift scores by up to almost two points. He released SacreBLEU, which handles reference processing internally and prints a signature of its settings; it is now the standard way to report BLEU.

chrF (2015) and chrF++ (2017)

Maja Popović proposed an F-score over character n-grams that needs no tokeniser or language-specific resources and suits morphologically rich targets. chrF++ added word unigrams and bigrams. Both are widely reported alongside BLEU.

COMET (2020)

Rei, Stewart, Farinha and Lavie trained a regression model over cross-lingual pretrained encodings of the source, hypothesis and reference to predict human quality scores. COMET marked the shift from string matching to learned metrics; later versions such as COMET-22 and xCOMET added error-span detection.

Improved Statistical Alignment Models (2000)

Och and Ney introduced evaluation of word alignment against human annotation with "sure" and "possible" links and the alignment error rate, and used it to compare the IBM models and the HMM aligner systematically. Their training refinements became the GIZA++ toolkit, used to align parallel corpora for more than a decade.

Discriminative Training and Maximum Entropy Models (2002)

Och and Ney replaced the equal-weight product of the noisy channel with a log-linear combination of arbitrary features whose weights are learned. This made statistical MT extensible: any new knowledge source could be added as a feature, and the framework underlay phrase-based and syntax-based systems until the neural era.

A Hierarchical Phrase-Based Model (2005)

David Chiang allowed phrase pairs to contain gaps filled by other phrases, forming a synchronous context-free grammar learned from unannotated bitext. The Hiero model captured long-distance reordering without a parser and outperformed a strong phrase-based system on Chinese–English. It became a standard alternative to flat phrase-based translation.

Recurrent Continuous Translation Models (2013)

Kalchbrenner and Blunsom translated using only continuous vectors: a convolutional sentence encoder conditioned a recurrent target language model, with no phrase tables or explicit alignments. It was among the first demonstrations that a purely neural model could learn translation, a year before the encoder–decoder papers.

Sequence-Level Knowledge Distillation (2016)

Kim and Rush trained compact students on the beam-search outputs of large teachers. The students came close to teacher quality, decoded well without beam search, and could be pruned heavily. Sequence-level distillation became a standard tool, notably for training non-autoregressive models.

Achieving Human Parity on Chinese to English News Translation (2018)

Hassan and colleagues at Microsoft defined human parity as a statistically insignificant difference between human judgments of machine and professional translations, and reported reaching it on WMT 2017 Chinese–English news. The claim provoked careful re-examination: Läubli, Sennrich and Volk and Toral and colleagues showed that document context, professional raters and original-language source text change the verdict.

Experts, Errors, and Context (2021)

Freitag and colleagues had professional translators annotate errors in top WMT 2020 systems using the Multidimensional Quality Metrics (MQM) framework, with full document context. The expert rankings differed sharply from crowd ratings, clearly favoured human translation, and agreed better with embedding-based metrics than with crowd scores. MQM annotation has since become the reference for metric evaluation at WMT.

Massively multilingual NMT (2019)

Arivazhagan and colleagues trained one Transformer on more than a hundred languages paired with English, using web-scale data with highly unequal amounts per language. Low-resource languages gained through transfer, while high-resource ones suffered from interference unless capacity grew — a trade-off that still frames multilingual modelling.

Unsupervised Machine Translation Using Monolingual Corpora Only (2017)

Lample, Conneau, Denoyer and Ranzato trained a translator without any parallel sentences, starting from an unsupervised bilingual lexicon and alternating denoising with back-translation in a shared latent space. Together with the parallel work of Artetxe and colleagues, it opened a research line aimed at languages with no bitext.

People

• Philipp Koehn, now at Johns Hopkins University, is the most prolific author on this topic. His papers here range from factored translation models and domain adaptation to the Edinburgh IWSLT system and many years of WMT findings; he led the development of Moses.
• Ondřej Bojar of Charles University is first author of most WMT findings papers of the 2010s and a central organiser of the shared tasks.
• Barry Haddow of the University of Edinburgh co-authored the WMT findings and a survey of low-resource MT.
• Franz Josef Och and Hermann Ney of RWTH Aachen University laid the foundations of phrase-based statistical MT with the alignment template approach, alignment models and log-linear training; Och later led translation work at Google.
• Kyunghyun Cho, now at New York University, co-authored the founding papers of neural MT — attention, encoder–decoder analysis, large vocabularies and multilingual translation — with Yoshua Bengio and Dzmitry Bahdanau.
• Rico Sennrich of the University of Zurich contributed subword and back-translation methods through the Edinburgh WMT systems, politeness control and work on context-aware translation.
• Alon Lavie of Carnegie Mellon University co-created METEOR and later COMET, bridging two generations of metrics.
• Ricardo Rei and André F. T. Martins, working in Lisbon, developed the COMET family of learned metrics; Martins also co-authored the Marian toolkit.
• Markus Freitag, now at ETH Zurich, led the expert MQM study, the WMT metrics tasks of 2022 and 2023, and work on references as a source of metric error.
• Lucia Specia of Imperial College London pioneered quality estimation and multimodal translation.
• Kevin Knight introduced syntax-based statistical translation and scaled up syntactic rule extraction with Daniel Marcu.
• Chris Callison-Burch showed how to mine paraphrases from bilingual corpora and applied translation methods to simplification.
• Orhan Firat is a leading author on multilingual NMT, from shared attention to massively multilingual models at Google.
• Matt Post, now at Microsoft, wrote the SacreBLEU paper and work on constrained decoding.
• Graham Neubig of Carnegie Mellon University contributed to rapid adaptation to new languages and discrete lexicons in NMT.

Other frequent contributors include Christian Federmann, Christof Monz of the University of Amsterdam, Yvette Graham of Trinity College Dublin, Alexandra Birch, Tom Kocmi, George Foster, Marcello Federico, Marcin Junczys-Dowmunt and Rachel Bawden, whose work on discourse phenomena helped establish document-level evaluation.

Institutions and places

The authors on this topic are concentrated in a handful of research centres that have shaped MT for decades. Among current affiliations, Microsoft leads, followed by the University of Edinburgh — home of the Moses project and of successive WMT-winning systems — Carnegie Mellon University, Johns Hopkins University and Charles University in Prague, which hosts much of the WMT organisation. Other important centres are the University of Amsterdam, Google, ETH Zurich and the University of Zurich, Imperial College London, and a Lisbon cluster (Instituto de Telecomunicações, Instituto Superior Técnico, University of Lisbon) associated with learned evaluation metrics. Historically, IBM Research, RWTH Aachen, the Information Sciences Institute at the University of Southern California and Fondazione Bruno Kessler in Trento were also major sources of work.

By country, the United States accounts for the largest share of authors, followed by the United Kingdom, China, Germany and Switzerland; Czechia, Canada and the Netherlands are also prominent, reflecting the European strength of the field built around Europarl, EU-funded projects and the WMT campaign.

The papers appear mostly in the ACL family of venues: the annual Conference (formerly Workshop) on Machine Translation, the main ACL and EMNLP conferences, Transactions of the ACL and the journal Computational Linguistics. Language-resource papers often appear at LREC, and the European Association for Machine Translation conference and dedicated workshops on low-resource MT and spoken language translation are well represented.

Open questions

• Evaluation at high quality. When systems approach or exceed reference quality, reference-based metrics lose meaning. How to evaluate reliably — with expert error annotation, source-based judgments or learned metrics — and how to avoid metrics that can be gamed remain open; recent WMT metrics tasks ask whether LLM output breaks existing metrics.
• Human parity and its meaning. Parity claims depend heavily on the evaluation protocol, rater expertise, document context and the direction in which the test set was originally written.
• Low-resource and endangered languages. Most of the world's languages still lack adequate parallel data. Community-driven work such as participatory research for African languages, transfer between related languages and evaluation sets such as FLORES are early steps.
• Document and discourse. Sentence-by-sentence translation mishandles pronouns, ellipsis and lexical cohesion; context-aware models and hierarchical attention over documents help, but evaluation of document-level quality is still immature.
• Hallucination and robustness. Neural systems can produce fluent text unrelated to the source; see the curious case of hallucinations and hallucinations in large multilingual models.
• Bias. Systems often default to stereotyped gender when the source is ambiguous, as shown by WinoMT and surveyed in Gender Bias in Machine Translation.
• Multilingual capacity. How to add languages without degrading existing ones, and how to allocate model capacity across hundreds of languages.
• The role of general LLMs. Whether dedicated translation models will remain competitive with general large language models, and how to control terminology, style and faithfulness in prompted translation.

Neighbouring topics

Machine translation sits at a crossroads. Its largest overlap is with Evaluation & benchmarks: metrics and human evaluation protocols developed for MT became templates for summarisation, generation and dialogue. Multilingual & cross-lingual NLP and Low-resource languages share its concern with data scarcity and transfer across languages. Work on Arabic and Hebrew translation, including Arabic dialects and Arabic preprocessing, links the topic to Semitic languages and Hebrew & Aramaic.

Speech translation connects it to Speech & spoken language, and multimodal translation to Vision & language. Linguistic structure enters through Morphology, Syntax & grammar formalisms, Tokenization & segmentation, Word sense disambiguation — as in WSD for statistical MT — and Idioms & multiword expressions. Document-level translation draws on Coreference resolution and Discourse analysis. Synchronous grammars link it to Semantic parsing & meaning representation. Neural MT gave rise to much of modern Language models & pretraining and Text generation, and questions of fairness tie it to Bias, fairness & safety. Related monolingual tasks such as paraphrasing and simplification, and translation-inspired approaches to Grammatical error correction, also borrow its methods.

Within the library, most papers on this topic also stand on the Machine translation shelf, with many on Linguistics & languages and Natural language processing.

Key works in this library

• BLEU — the n-gram precision metric that defined MT evaluation for two decades.
• Jointly Learning to Align and Translate — introduced attention for translation.
• On the Properties of Neural Machine Translation — diagnosed the length and vocabulary limits of early encoder–decoders.
• A Statistical Approach to Language Translation — the IBM proposal that launched statistical MT.
• Aligning Sentences in Bilingual Corpora — length-based sentence alignment.
• Improved Statistical Alignment Models — alignment error rate and a systematic comparison of alignment models.
• Discriminative Training and Maximum Entropy Models — the log-linear framework.
• A Hierarchical Phrase-Based Model for SMT — synchronous grammars learned without a parser.
• A Syntax-based Statistical Translation Model — tree-to-string translation.
• Factored Translation Models — linguistic annotation inside phrase-based MT.
• Large Language Models in Machine Translation — trillion-token n-gram models and stupid backoff.
• METEOR — alignment-based metric with stems and synonyms.
• chrF — character n-gram F-score.
• A Call for Clarity in Reporting BLEU — SacreBLEU and reproducible scoring.
• COMET — a learned neural metric.
• Experts, Errors, and Context — MQM expert evaluation of top systems.
• OpenNMT — open-source neural MT toolkit.
• Sequence-Level Knowledge Distillation — compressing translation models.
• Modeling Coverage for Neural Machine Translation — tracking which source words have been translated.
• Scaling Neural Machine Translation — fast large-batch Transformer training.
• Non-Autoregressive Neural Machine Translation — parallel decoding.
• Achieving Human Parity (Chinese–English news) — the human-parity claim and its protocol.
• Unsupervised Machine Translation Using Monolingual Corpora Only — translation without parallel data.
• Massively Multilingual NMT in the Wild — one model for over a hundred languages.
• Evaluating Gender Bias in Machine Translation — the WinoMT challenge set.