Elamur Research⌕
☰
home / papers / 2026 / NRD: A Hybrid Disentanglement Framework for Mitigating Interference in Multilingual Machine Translation

NRD: A Hybrid Disentanglement Framework for Mitigating Interference in Multilingual Machine Translation

Summary

Multilingual NMT systems that share one encoder across many languages often degrade when typologically distant languages push conflicting syntactic patterns into the same representations. The authors attribute this negative interference to meaning and language-specific structure being mixed within the same neurons. They note that earlier neuron-level remedies act either only during training or only during decoding. NRD (Neuron Representation Disentanglement) connects both phases through a single neuron partition in the encoder FFN layers. First, each neuron is classified by how much its activation varies across meaning-preserving paraphrases produced by controlled back-translation. Stable units count as semantic and fluctuating ones as syntax-sensitive, and fine-tuning then updates these groups sparsely. Second, at inference, a language-specific mask dampens syntax-sensitive neurons, with its strength set by each language's precomputed gradient consistency, so no per-language retraining is required. The paper reports average gains of +1.9 BLEU over strong baselines on supervised OPUS-100 directions and +7.1 BLEU on WMT-10 zero-shot translation. This suggests cleaner, more transferable representations, which the authors see as promising for low-resource languages.

Key points

  • Two-stage method: Specialization Fine-tuning (variance-based neuron identification plus sparse updates) and Dynamic Representation Filtering (gradient-consistency-controlled masking at inference).
  • "Neurons" are individual hidden units of the FFN intermediate layer in Transformer encoder layers. Paraphrase sets are built once per language and reused.
  • Results: +1.9 BLEU average on OPUS-100 supervised directions and +7.1 BLEU on the WMT-10 zero-shot benchmark.
  • Positioned against neuron specialization (Tan et al., 2024), CaPA gradient-based capacity allocation (Huo et al., 2024), and universal pruning (Kim et al., 2024). Adapters (Pfeiffer et al., 2020) are cited among parameter-isolation approaches.
  • Performance depends on paraphrase quality (the paper's Table 4). Planned extensions are LLM pre-training, other cross-lingual tasks, and automated neuron identification.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/55wnhwvmezwx

@inproceedings{zhang2026nrd,
  title = {{NRD: A Hybrid Disentanglement Framework for Mitigating Interference in Multilingual Machine Translation}},
  author = {Zhang, Jiarui and Deng, Yifan},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/55wnhwvmezwx},
  url = {https://doi.org/10.63317/55wnhwvmezwx}
}

References

16 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. arXiv preprint arXiv:1903.00089. find our card
  2. Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, et al. 2019. Massively multilingual neural machine translation in the wild: Findings and challenges. arXiv preprint arXiv:1907.05019. find our card
  3. Ankur Bapna and Orhan Firat. 2019. Simple, scalable adaptation for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1538–1548. find our card
  4. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  5. Wenshuai Huo, Xiaocheng Feng, Yichong Huang, Chengpeng Fu, Hui Wang, and Bing Qin. 2024. Gradient consistency-based parameter allocation for multilingual neural machine translation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluat… find
  6. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational L… find
  7. Hwichan Kim, Jun Suzuki, Tosho Hirasawa, and Mamoru Komachi. 2024. Pruning multilingual large language models for multilingual inference. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9921–9942. find
  8. Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, et al. 2020. Xlm-t: Scaling up multilingual machine translation with pretrained cross-lingual transformer encoders. arXiv preprint arXiv:2012.15547. — [arXiv:2012.15547](https://arxiv.org/abs/2012.15547) find
  9. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr… find our card
  10. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterhub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, p… find our card
  11. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics. find our card
  12. Shaomu Tan, Di Wu, and Christof Monz. 2024. Neuron specialization: Leveraging intrinsic task modularity for multilingual machine translation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6506–6527. find
  13. Yiren Wang, ChengXiang Zhai, and Hany Hassan. 2020a. Multi-task learning for multilingual neural machine translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1022–1034, Online. Association for Computational Linguistics. find
  14. Zirui Wang, Zachary C Lipton, and Yulia Tsvetkov. 2020b. On negative interference in multilingual models: Findings and a meta-learning treatment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4438–4450. find our card
  15. Wanying Xie, Yang Feng, Shuhao Gu, and Dong Yu. 2021. Importance-based neuron allocation for multilingual neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing… find
  16. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628–1639. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-nrd-hybrid-disentanglement-framework-mitigating/