非自回归神经机器翻译综述

Summary

This survey from the NLP Lab at Northeastern University (Shenyang) reviews non-autoregressive neural machine translation (NART). Standard autoregressive models (ART) generate target words one at a time, which makes inference slow. NART instead predicts all positions in parallel under a conditional-independence assumption, so it decodes much faster. The cost is that the model ignores dependencies between target words, which leads to repeated or missing tokens and lower quality. The authors identify three core difficulties: choosing the decoder input, predicting the output length in advance, and the multimodality problem, where one source sentence has several valid translations. Unlike an earlier modular survey, they group methods by how each one recovers target-side dependencies. For each group they discuss strengths and weaknesses, and they close with open research directions. The paper contains no new experiments. Its contribution is a structured overview of the field plus a curated, publicly released paper list labelled by method, venue and year. This makes it a convenient entry point for anyone working on faster MT decoding.

Key points

• Six method families:
  • training-data improvement, mainly sequence-level knowledge distillation from an ART teacher;
  • iterative refinement, covering latent-variable iteration, insertion/deletion (e.g., Stern et al., Levenshtein-style), and mask-predict with CMLM;
  • latent-variable models, such as fertility in Gu et al.'s NAT, Latent Transformer, VQ-VAE, position/reordering/alignment latents, and flow-based models;
  • modified training objectives, such as CTC, CRF, reinforcement/sequence-level losses, and regularizers;
  • ART-assisted learning;
  • other approaches, including semi-autoregressive models and training strategies.
• Role of distillation: it reduces lexical diversity and reordering complexity in the training data. Teacher-data complexity must match the NART model's capacity, and a larger teacher is not automatically better. Distillation can also propagate errors on low-frequency words.
• Iterative methods: some approach or even exceed ART quality. However, they give up much of the speed advantage, since an ART model with a deep encoder and a single-layer decoder can be comparably fast.
• Latent-variable methods: they shrink the search space but are hard to design and add computational overhead.
• Resource and publication details: an annotated NART paper collection is hosted on GitHub (NAT-survey). The PDF header shows vol. 37, no. 11 (Nov 2023), pp. 1–14.