Elamur Research
☰
home / papers / 2018 / Getting Gender Right in Neural Machine Translation

Getting Gender Right in Neural Machine Translation

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D18-1334.pdf

Summary

English often leaves the speaker's gender unmarked. Many target languages instead require gendered forms, such as a French adjective agreeing with a female speaker. MT systems usually lack this information and tend to default to the statistically dominant, often male, variant. The authors address this in two ways. First, they release Europarl-based parallel corpora for 20 English-centred language pairs, annotated with speaker metadata (gender, age and more) by linking Europarl speaker tags to MEP information from Rabinovich et al. (2017). Second, for the 10 pairs with more than 500k annotated sentences, they train OpenNMT-py LSTM encoder-decoders with and without a speaker-gender tag prepended to the English source sentence.

On a general test set, the tagged systems significantly beat the baselines for French, Italian and Danish. Portuguese and Greek improve slightly, while Spanish, German, Dutch, Swedish and Finnish do not improve. Manual inspection of EN–FR output shows that the gains come from better gender agreement and also from word choices that differ between male and female speakers. This is the first speaker-informed NMT study and offers a simple way to reduce gender errors and bias.

Key points

  • The released resource covers 20 language pairs with Europarl speaker annotations (name, gender, age, date of birth, euroID, session date). It is available under CC BY 4.0 on GitHub as Europarl-Speaker-Information, with sizes ranging from about 252k (EN–HU) to 1.44M (EN–FR) sentences.
  • The data is gender-imbalanced. Only about 32–33% of sentences come from female speakers. In EN–FR, 67.39% of sentences are male, and the 50–60 age group produces 43.76% of all sentences.
  • Setup:
    • OpenNMT-py with default parameters
    • BPE with 89,500 merge operations
    • 13 training epochs
    • a 2K random test set, plus male-only and female-only test sets for EN–FR
  • BLEU results, baseline vs. tagged:
    • Significant gains: FR 37.82 → 39.26, IT 31.46 → 31.75, DA 36.69 → 37.00.
    • Small, non-significant gains: PT 36.11 → 36.33, EL 31.38 → 31.54.
    • No gain: ES fell slightly (42.47 → 42.28), and FI dropped significantly (21.82 → 21.35).
  • In the qualitative EN–FR analysis, the tagged system fixed agreement errors such as heureux → heureuse, but not every case. It also preferred lexical variants like pense over crois, which fits corpus evidence that "crois" is relatively more frequent among male speakers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

127 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 120 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1334

@inproceedings{vanmassenhove2018getting,
  title = {{Getting Gender Right in Neural Machine Translation}},
  author = {Vanmassenhove, Eva and Hardmeier, Christian and Way, Andy},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1334},
  url = {https://doi.org/10.18653/v1/d18-1334}
}

References

22 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. In International Conference on Learning Representations, Banff, Canada. find our card
  2. Kyunghyun Cho, Bart van Merriënboer, Çalar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder– Decoder for Statistical Machine Translation. In Proceedings of EMNLP 2014, pages 1724–1734, Doha, Qatar. find
  3. Jonathan H Clark, Chris Dyer, Alon Lavie, and Noah A Smith. 2011. Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short paper… find
  4. Jennifer Coates. 2015. Women, Men and Language: A Sociolinguistic Account of Gender Differences in Language. Routledge, London. find
  5. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. Transactions of the Association of Computational L… find
  6. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-Source Toolkit for Neural Machine Translation. In Proceeding of ACL, Vancouver, Canada. find
  7. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In MT Summit, volume 5, pages 79–86, Phuket, Thailand. find
  8. Tatiana Litvinova, Pavel Seredin, Olga Litvinova, Olga Zagorovskaya, Aleksandr Sboev, Dmitry Gu-dovskih, Ivan Moloshnikov, and Roman Rybka. 2016. Gender Prediction for Authors of Russian Texts Using Regression And Classification Techniques. In Proceedings of the Third Workshop on Concept Discovery i… find
  9. Shachar Mirkin, Scott Nowson, Caroline Brun, and Julien Perez. 2015. Motivating Personality-Aware Machine Translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1102–1108, Lisbon, Portugal. find
  10. Matthew L Newman, Carla J Groom, Lori D Handelman, and James W Pennebaker. 2008. Gender Differences in Language Use: An Analysis of 14,000 Text Samples. Discourse Processes, 45(3):211–236. of the 40th annual meeting on association for computational linguistics, pages 311–318. find
  11. James W. Pennebaker, Matthias R. Mehl, and Kate G. Niederhoffer. 2003. Psychological Aspects of Natural Language Use: Our words, Our Selves. Annual review of psychology, 54(1):547–577. find
  12. Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. Personalized Machine Translation: Preserving Original Author Traits. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 10… find
  13. Francisco Rangel, Paolo Rosso, Moshe Koppel, Efstathios Stamatatos, and Giacomo Inches. 2013. Overview of The Author Profiling Task at PAN 2013. In CLEF Conference on Multilingual and Multimodal Information Access Evaluation, pages 352–365. find
  14. K Santosh, Romil Bansal, Mihir Shekhar, and Vasudeva Varma. 2013. Author profiling: Predicting age and gender from blogs. Notebook for PAN at CLEF, pages 119–124. find
  15. Edward Sapir. 1921. Language: An Introduction to the Study of Speech. NewYork: Harcourt Brace & Company. find
  16. Rico Sennrich. 2015. Modelling and Optimizing on Syntactic N-grams for Statistical Machine Translation. Transactions of the Association for Computational Linguistics, 3:169–182. find
  17. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling Politeness in Neural Machine Translation via Side Constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, Berlin, G… find our card
  18. Dan I. Slobin. 1996. From Thought and Language to Thinking for Speaking. In J. Gumperz and S. Levinson, editors, Rethinking Linguistic Relativity, pages 70–96. Cambridge University Press, Cambridge. find
  19. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, pages 3104–3112, Montreal, Quebec, Canada. find
  20. Deborah Tannen. 1991. You Just Don’t Understand. Ballantine Books, New York, USA. find
  21. Eva Vanmassenhove and Christian Hardmeier. 2018. Europarl Datasets with Demographic Speaker Information. In EAMT, Alicante, Spain. find
  22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings find our card