📄 Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection

Summary

Neural machine translation systems sometimes produce "detached hallucinations," which are outputs largely unrelated to the source sentence. These errors are rare in natural text, so it has been hard to study when they occur, and most existing detectors treat the model as a black box. The authors look inside a Transformer NMT model using Layerwise Relevance Propagation (LRP), which measures how much each source and target token contributes to every prediction. They built an English→Chinese counterfactual dataset by perturbing source sentences and pairing hallucinated outputs with normal translations of the unperturbed inputs. On this data they tested three hypotheses about hallucination symptoms. Two source-side patterns proved to be the clearest signals: contributions concentrated on a few source tokens, and a contribution distribution that barely changes across decoding steps. The relative weight of source versus target context mattered less. Features built from these patterns feed a lightweight classifier. On manually annotated English-Chinese and German-English hallucinations, it beats model-free baselines, quality-estimation-based detectors, and black-box detectors built on large pre-trained models, and it holds up better under domain shift.

Key points

• Perturbations used:
  • character deletion (misspelling) with probability 0.1;
  • random title-casing with probability 0.1;
  • inserting one token at the start of the sentence, drawn from frequent, rare, or mid-frequency tokens, or punctuation.
• Data construction: 50k seed sentence pairs were sampled from the training corpora. Hallucinations were flagged with BLEU-based heuristics:
  • the original translation must be close to the reference (BLEU > 0.3);
  • the perturbed-input output must be far from the original translation (BLEU < 0.03);
  • the perturbed-input output must not simply copy the source (BLEU < 0.5).
• Resulting dataset: 623 contrastive pairs for misspelling, 270 for title-casing, and 1307 for insertion.
• Hypotheses tested: Low Source Contribution (Voita et al.), Local Source Contribution (Lee et al.), and a newly formalized Static Source Contribution hypothesis. The two source-based patterns were stronger indicators than low source contribution.
• Resources and limitations: The best detector, LRP + LASER, also looked promising in a small stress test where hallucinations were rare. The study covers only detached hallucinations and WMT-trained models. Code and data are available at github.com/weijia-xu/hallucinations-in-nmt.