Elamur Research⌕
☰
home / papers / 2026 / Attribute-Controlled Translation with Preference Optimization

Attribute-Controlled Translation with Preference Optimization

Paper · 2026 · Inigo Jauregi Unanue, Najmeh Sadoughi, Vimal Bhat et al.

Summary

Attribute-controlled translation (ACT) asks a system to translate accurately while also respecting a requested property of the output, such as formality or grammatical gender. Strong results have mostly come from prompting very large LLMs, and smaller models are held back by scarce training data. The authors note that ACT datasets already pair each source sentence with two references of opposite attributes, which maps naturally onto preference learning. They fine-tune smaller models with contrastive preference optimization (CPO), a lighter variant of DPO.

Plain attribute-contrastive pairs improve attribute control but reduce translation quality, because the two references differ only in their attribute markers. To fix this, the authors use larger LLMs to generate synthetic preference pairs that differ in both attribute compliance and quality, which works as a form of knowledge distillation. They also release PREF-FAME-MT, a contrastive formality corpus built from FAME-MT.

Across three datasets and 71 language pairs, the combined method improves attribute matching while largely preserving quality. In many cases it brings models such as NLLB and Qwen3 8B close to a state-of-the-art LLM.

Key points

  • Datasets and models: Evaluation uses CoCoA-MT and PREF-FAME-MT for formality and MT-GenEval for gender, covering 71 language pairs in total. The models tested are NLLB, Qwen3 8B and EuroLLM 9B.
  • Building PREF-FAME-MT: The authors expanded FAME-MT (en→de, es, fr, hi, it, ja, nl, pt) by prompting LLMs to restyle target references, which produced contrastive formal/informal pairs.
    • Many English source sentences are shared across languages, so they served as pivots to create 56 additional non-English pairs.
    • The test set was validated by humans, and the zero-shot results suggest it is harder than CoCoA-MT.
  • NLLB results on PREF-FAME-MT:
    • Instruction tuning on the synthetic data, compared with tuning on the original data, raised M-Acc by +5.5 pp and BLEU by +1.37.
    • CPO on top of instruction tuning added +6.1 pp M-Acc but lowered KIWI-XXL by −3.7.
    • CPO with synthetic data recovered COMET and KIWI-XXL while still slightly improving attribute control.
  • Other findings:
    • The decoder-only models kept their translation quality under CPO. Qwen3 was better at attribute control, and EuroLLM was slightly better at translation quality.
    • Using more synthetic data improved every metric.
    • Hindi pairs gained less, because fewer of their synthetic samples passed the validity checks.
  • Limitations and availability: Gender is treated as binary only, and the method may inherit biases from the LLMs that generate the data. Code and data are available at github.com/inigo-jauregi/act-po.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AmazonSeattle · United States112 works · 40,565 citations🎓University of Technology SydneySydney · Australia18 works · 12,833 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Inigo Jauregi Unanue, Najmeh Sadoughi, Vimal Bhat, Zhu Liu and Massimo Piccardi. 2026. Attribute-Controlled Translation with Preference Optimization. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.209

@inproceedings{unanue2026attribute,
  title = {{Attribute-Controlled Translation with Preference Optimization}},
  author = {Inigo Jauregi Unanue, Najmeh Sadoughi, Vimal Bhat, Zhu Liu and Massimo Piccardi},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.209},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.209}
}

References

42 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. https://www-cdn.anthropic.com/ de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/ Model_Card_Claude_3.pdf. Model Card. ↗
  2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  3. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 4302–4310, Red Hook, NY, USA. Curran Associa… find
  4. Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  5. Marta R. Costa-jussà, Pau Li Lin, and Cristina España-Bonet. 2020. GeBioToolkit: Automatic extraction of gender-balanced multilingual corpus of Wikipedia biographies. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4081–4088, Marseille, France. European Language Res… find
  6. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. Ultrafeedback: boosting language models with scaled ai feedback. In Proceedings of the 41st International Conference on Machine Learning, ICML’24.… find
  7. Guofeng Cui, Pichao Wang, Yang Liu, Zemian Ke, Zhu Liu, and Vimal Bhat. 2025. CRPO: Confidence-reward driven preference optimization for machine translation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 560–574, Vienna, Austria. Association for Computational Linguist… find
  8. Anna Currey, Maria Nadejde, Raghavendra Reddy Pappagari, Mia Mayer, Stanislas Lauly, Xing Niu, Benjamin Hsu, and Georgiana Dinu. 2022. MT-GenEval: A counterfactual and contextual dataset for evaluating gender accuracy in machine translation. In Proceedings of the 2022 Conference on Empirical Methods… find
  9. Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. The hitchhiker’s guide to testing statistical significance in natural language processing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1383–1392, Melbourne, A… find
  10. Scott Geng, Hamish Ivison, Chun-Liang Li, Maarten Sap, Jerry Li, Ranjay Krishna, and Pang Wei Koh. 2025. The delta learning hypothesis: Preference tuning on weak data can yield strong gains. arXiv preprint arXiv:2507.06187. find
  11. D. S. Hirschberg. 1975. A linear space algorithm for computing maximal common subsequences. Commun. ACM, 18(6):341–343. find
  12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  13. Surafel Melaku Lakew, Mattia Di Gangi, and Marcello Federico. 2019. Controlling the output length of neural machine translation. In Proceedings of the 16th International Conference on Spoken Language Translation, Hong Kong. Association for Computational Linguistics. find
  14. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024a. Rlaif vs. rlhf: scaling reinforcement learning from human feedback with ai feedback. In Proceedings of the 41st International… find
  15. Minwoo Lee, Hyukhun Koh, Minsung Kim, and Kyomin Jung. 2024b. Fine-grained Gender Control in Machine Translation with Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Lo… find
  16. Danni Liu and Jan Niehues. 2024. How transferable are attribute controllers on pretrained multilingual translation models? In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 334–348, St. Julian’s, Malta. Assoc… find
  17. Pedro Henrique Martins, João Alves, Patrick Fernandes, Nuno M Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M Alves, José Pombal, Nicolas Boizard, and 1 others. 2025. Eurollm-9b: Technical report. arXiv preprint arXiv:2506.04079.
  18. Paul Michel and Graham Neubig. 2018. Extreme adaptation for personalized neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 312–318, Melbourne, Australia. Association for Computational Linguistics.
  19. Maria Nadejde, Anna Currey, Benjamin Hsu, Xing Niu, Marcello Federico, and Georgiana Dinu. 2022. CoCoA-MT: A dataset and benchmark for contrastive controlled MT with application to formality. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 616–632, Seattle, United Sta… find
  20. Xing Niu, Sudha Rao, and Marine Carpuat. 2018. Multi-task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1008–1021, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  21. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  23. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  24. Spencer Rarrick, Ranjita Naik, Varun Mathur, Sundar Poudel, and Vishal Chowdhary. 2023. Gate: A challenge set for gender-ambiguous translation examples. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’23, page 845–854, New York, NY, USA. Association for Computing Mac… find
  25. Ricardo Rei, Nuno M. Guerreiro, José Pombal, Daan van Stigt, Marcos Treviso, Luisa Coheur, José G. C. de Souza, and André Martins. 2023. Scaling up CometKiwi: Unbabel-IST 2023 submission for the quality estimation shared task. In Proceedings of the Eighth Conference on Machine Translation, pages 84…
  26. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  27. Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. ChatGPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Association for Computational Linguistics.
  28. Gabriele Sarti, Phu Mon Htut, Xing Niu, Benjamin Hsu, Anna Currey, Georgiana Dinu, and Maria Nadejde. 2023. RAMP: Retrieval and attribute-marking enhanced prompting for attribute-controlled translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volum… find
  29. Danielle Saunders, Rosie Sallis, and Bill Byrne. 2020. Neural machine translation doesn’t translate gender coreference right unless you make it. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 35–43, Barcelona, Spain (Online). Association for Computational…
  30. Andrea Schioppa, David Vilar, Artem Sokolov, and Katja Filippova. 2021. Controlling machine translation for multiple attributes with additive interventions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6676–6696, Online and Punta Cana, Dominican Re… find
  31. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  32. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, San Diego…
  33. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, US… find
  34. Dawid Wisniewski, Zofia Rostek, and Artur Nowakowski. 2024. FAME-MT dataset: Formality awareness made easy for machine translation purposes. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 164–180, Sheffield, UK. European Association… find
  35. Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024a. Contrastive preference optimization: pushing the boundaries of llm performance in machine translation. In Proceedings of the 41st International Conference on Machine Learning,…
  36. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. 2024b. Is dpo superior to ppo for llm alignment? a comprehensive study. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find
  37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  38. Sho Takase and Naoaki Okazaki. 2019. Positional encoding to control output sequence length. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3999–4004, Minneapoli… find
  39. Kaden Uhlig, Joern Wuebker, Raphael Reinauer, and John DeNero. 2024. Cross-lingual human-preference alignment for neural machine translation with direct quality optimization. arXiv preprint arXiv:2409.17673. find
  40. Dario Vajda, Domen Vreš, and Marko Robnik-Šikonja. 2025. Improving llms for machine translation using synthetic preference data. arXiv preprint arXiv:2508.14951. find
  41. Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting gender right in neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3003–3008, Brussels, Belgium. Association for Computational Linguistics.
  42. Yifan Wang, Zewei Sun, Shanbo Cheng, Weiguo Zheng, and Mingxuan Wang. 2023. Controlling styles in neural machine translation with activation prompt. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2606–2620, Toronto, Canada. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/unanue-2026-attribute-controlled-translation-preference-optimization/