Elamur Research⌕
☰
home / papers / 2026 / One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment

One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment

Paper · 2026 · Hao Zong, Cong Hu Yuan, Chao Bei et al.

Summary

Large language models translate fluently but perform poorly on low-resource languages. Specialized multilingual encoders such as NLLB cover many languages but cannot generate as flexibly. The authors argue that the obstacle is aligning the two representation spaces, not a shortage of training data. They propose Hybrid Cross-Alignment (HCA), which links a frozen NLLB encoder to a frozen Qwen3 (4B) decoder through three trainable components:

  • a Source-Side Alignment Adapter inside the encoder that reshapes its features,
  • a Cross-Modal Alignment Adapter that injects those features into every decoder layer through gated cross-attention,
  • a Query-Residual Adapter that keeps the LLM's generation stable.

HCA is trained only on German-English, yet it translates dozens of unseen languages into English. On low-resource languages it beats larger systems such as TowerPlus-9B and Aya-101. The authors call this effect "Source-Side Alignment Generalization." The result suggests that one modest bilingual dataset can unlock existing encoder knowledge for X-to-English translation. Limitations: quality is capped by what the encoder already knows, and the English-to-X direction remains untested.

Key points

  • The full system has about 5.25B parameters. It was trained for one epoch on roughly 580K De-En pairs, using under 1% of the compute needed for full fine-tuning.
  • On low-resource languages, the zero-shot model averages 82.6 COMET, compared with 73.6 for TowerPlus-9B and 80.7 for Aya-101 (13B). The chrF++ results show the same pattern.
  • Training on one pair recovers over 96.7% of the performance of an "Oracle" model trained on all pairs (82.9 vs. 85.8 COMET). The Oracle slightly outperforms the NLLB-1.3B teacher (85.6).
  • Ablations show that removing the Query-Residual Adapter collapses performance by about 26 points, falling below Qwen3 zero-shot. Removing the Source-Side Adapter costs about 1–2 points.
  • Replacing the 600M encoder with the 1.3B one gives gains such as +3.4 COMET on Gujarati without retraining the LLM. Training on Arabic-English instead of De-En also generalizes well.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Beijing Jiaotong UniversityBeijing · China28 works · 1,754 citations🎓Dalian University of TechnologyDalian · China20 works · 46 citations🎓Fuyao University of Science and TechnologyFuzhou · China2 works

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Low-resource languages · 287 works

Cite

Cite as Hao Zong, Cong Hu Yuan, Chao Bei, Wentao Chen, Huan Liu, Kaiyu Huang and Degen Huang. 2026. One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1912

@inproceedings{zong2026one,
  title = {{One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment}},
  author = {Hao Zong, Cong Hu Yuan, Chao Bei, Wentao Chen, Huan Liu, Kaiyu Huang and Degen Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1912},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1912}
}

References

30 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. Tower: An open multilingual large language model for translation-related tasks.…
  2. Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Sriram Ganapathy, Abhishek Bapna, Prateek Jain, and Partha Talukdar. 2024. LLM augmented LLMs: Expanding capabilities through composition. In The Twelfth International Conference on Learning Representations. find
  3. Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, and 1 others. 2023. Seamlessm4t: massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596.
  4. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, pages 19730–19742.
  5. Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, and Furu Wei. 2021. Zero-shot cross-lingual transfer of neural machine translation with multilingual pretrained encoders. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pag… find
  6. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  7. Marta R Costa-jussà, James Tran, Artem Sokolov, Tripti Dewan, Guillaume Wenzek, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  9. Paul-Ambroise Duquenne, Holger Schwenk, and Benoît Sagot. 2023. Sonar: sentence-level multimodal and language-agnostic representations. arXiv preprint arXiv:2308.11466.
  10. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic bert sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891.
  11. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André F. T. Martins. 2022. Results of WMT22 metrics shared task: Stop using BLEU – neural metrics are better and more robust. In Proceedings of the Seventh Conferen…
  12. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning. find
  13. Sandhini Agarwal Lama Ahmad Ilge Akkaya Josh Achiam, Steven Adler and others. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  14. Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, and Arul Menezes. 2021. To ship or not to ship: An extensive evaluation of automatic metrics for machine translation. In Proceedings of the Sixth Conference on Machine Translation, pages 478–494, Online.… find
  15. Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, and Jingbo Zhu. 2025. Beyond decoder-only: Large language models can be good encoders for machine translation. Preprint, arXiv:2503.06594. find
  16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  17. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  18. Ricardo Rei, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F. T. Martins. 2025. Tower+: Bridging generality and translation specialization in multilingual llms. Preprint, arXiv:2506.17080. find
  19. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702.
  20. Fabian David Schmidt, Philipp Borchert, Ivan Vulić, and Goran Glavaš. 2024. Self-distillation for model stacking unlocks cross-lingual NLU in 200+ languages. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6724–6743, Miami, Florida, USA. Association for Computational… find
  21. Fanqi Wan, Longguang Zhong, Ziyi Yang, Ruijun Chen, and Xiaojun Quan. 2025. FuseChat: Knowledge fusion of chat models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21618–21642, Suzhou, China. Association for Computational Linguistics.
  22. Hao Wang, Linlong Xu, Heng Liu, Yangyang Liu, Xiaohu Zhao, Bo Zeng, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2025. Marco large translation model at wmt2025: Transforming translation capability in llms via quality-aware training and decoding. find
  23. In Proceedings of the Tenth Conference on Machine Translation, pages 587–593. find
  24. Haoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang, Akiko Eriguchi, and Huda Khayrallah. 2025. X-ALMA: Plug & play modules and adaptive rejection for quality translation at scale. In The Thirteenth International Conference on Learning Representations. find
  25. A Table 6 lists the ISO language codes, FLORES-200 identifiers, and full language names used in our experiments. We categorize them into High-Resource and Low-Resource groups consistent with our experimental setup. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Cha… — Qwen3 Technical Report
  26. Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. 2023. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199.
  27. Shaolei Zhang, Kehao Zhang, Qingkai Fang, Shoutao Guo, Yan Zhou, Xiaodong Liu, and Yang Feng. 2024. Bayling 2: A multilingual large language model with efficient language alignment. find
  28. Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, and Furu Wei. 2021. Allocating large vocabulary capacity for cross-lingual language model pre-training. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3203–3215. find
  29. Language Codes and Details Category Abbr. FLORES Code Language Source De En deu_Latn eng_Latn German English High-Res Zh Ru Ar zho_Hans rus_Cyrl arb_Arab Chinese Russian Arabic Low-Res Zu Xh My Lo Gu Prs zul_Latn xho_Latn mya_Mymr lao_Laoo guj_Gujr prs_Arab Zulu Xhosa Burmese Lao Gujarati Dari *Tabl… find
  30. Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D’souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Muennighoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. Aya model: An instruction fine…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zong-2026-one-pair-suffices-unlocking-universal/