Elamur Research⌕
☰
home / papers / 2020 / Does Gender Matter? Towards Fairness in Dialogue Systems

Does Gender Matter? Towards Fairness in Dialogue Systems

Paper · 2020 · Haochen Liu, Jamell Dacon, Wenqi Fan et al.

Summary

Dialogue systems learn from real human conversations, so they can pick up social prejudices. Their fairness had received little systematic study. The authors examine whether popular dialogue models treat genders (male vs. female) and races differently, with race represented as standard English vs. African American English. They frame fairness formally: a model is fair if a chosen response-quality score has equal expected value for a context and its parallel version, where group-related words are swapped for their counterparts. They build parallel context pairs by filtering a large dialogue corpus with curated gender and race word lists, then compare outcomes with a Z-test. Responses are scored on four dimensions: diversity, politeness, sentiment, and attribute words. Tests on a generative model and a Transformer retrieval model reveal statistically significant gender and racial biases. Two simple debiasing methods, CDA and WER, mostly remove these significant gaps, and WER is the stronger of the two. The released benchmark and code provide a basis for further fairness research in conversational AI.

Key points

  • Parallel contexts are created by swapping word pairs, such as "he"/"she" or "this"/"dis". Bias is flagged when the Z-test p-value falls below 0.05.
  • Diversity is the mean of distinct-1 and distinct-2. Politeness is the share of responses judged offensive by the classifier of Dinan et al. (2019b).
  • CDA enlarged the training data from about 2,580,433 to 4,197,883 single-turn dialogues. WER used a coefficient k = 0.5.
  • WER reduces bias more than CDA, but it can make responses to both groups overly similar and lower diversity. This points to a trade-off between performance and fairness.
  • The dataset and implementation are available at github.com/zgahhblhc/DialogueFairness.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong Polytechnic UniversityHong Kong · China34 works · 4,476 citations🎓Michigan State UniversityEast Lansing · United States7 works · 635 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSocial Sciences · 744 worksSafety Research · 221 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksBias, fairness & safety · 209 works

Cite

Cite as Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu and Jiliang Tang. 2020. Does Gender Matter? Towards Fairness in Dialogue Systems. Proceedings of the 28th International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.390

@inproceedings{liu2020does,
  title = {{Does Gender Matter? Towards Fairness in Dialogue Systems}},
  author = {Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu and Jiliang Tang},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.390},
  url = {https://doi.org/10.18653/v1/2020.coling-main.390}
}

References

44 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arturs Backurs, Piotr Indyk, Krzysztof Onak, Baruch Schieber, Ali Vakilian, and Tal Wagner. 2019. Scalable fair clustering. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pages 405–413. find
  2. Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, et al. 2018. Ai fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv pr… — [arXiv:1810.01943](https://arxiv.org/abs/1810.01943) find
  3. Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael J. Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. A convex framework for fair regression. CoRR, abs/1706.02409. — [arXiv:1706.02409](https://arxiv.org/abs/1706.02409) find
  4. Steven Bird. 2006. NLTK: the natural language toolkit. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006.
  5. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems…
  6. Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. CoRR, abs/1904.03035.
  7. Avishek Joey Bose and William Hamilton. 2019. Compositional fairness constraints for graph embeddings. CoRR, abs/1905.10674. — [arXiv:1905.10674](https://arxiv.org/abs/1905.10674) find
  8. Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017. A survey on dialogue systems: Recent advances and new frontiers. CoRR, abs/1711.01731. find
  9. Xingyu Chen, Brandon Fain, Liang Lyu, and Kamesh Munagala. 2019. Proportionally fair clustering. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pages 1032–1041. find
  10. Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2019a. Queens are powerful too: Mitigating gender bias in dialogue generation. arXiv preprint arXiv:1911.03842.
  11. Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019b. Build it break it fix it for dialogue safety: Robustness from adversarial human attack. CoRR, abs/1908.06083.
  12. Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multi-dimensional gender bias classification. CoRR, abs/2005.00614. find
  13. Jianfeng Gao, Michel Galley, and Lihong Li. 2019. Neural approaches to conversational AI. Foundations and Trends in Information Retrieval, 13(2-3):127–298. find
  14. Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, and Joelle Pineau. 2018. Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2018, New Orleans, LA, USA, February 02-… find
  15. Ayanna Howard and Jason Borenstein. 2018. The ugly truth about ourselves and our robot creations: the problem of bias and social inequity. Science and engineering ethics, 24(5):1521–1536. find
  16. Clayton J. Hutto and Eric Gilbert. 2014. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the Eighth International Conference on Weblogs and Social Media, ICWSM 2014, Ann Arbor, Michigan, USA, June 1-4, 2014. find
  17. Aylin Caliskan Islam, Joanna J. Bryson, and Arvind Narayanan. 2016. Semantics derived automatically from language corpora necessarily contain human biases. CoRR, abs/1608.07187. find
  18. Dan Jurafsky and James H. Martin. 2009. Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition. Prentice Hall series in artificial intelligence. Prentice Hall, Pearson Education International. find
  19. Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. Fairness-aware classifier with prejudice remover regularizer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 35–50. Springer. find
  20. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  21. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Sa…
  22. Haochen Liu, Tyler Derr, Zitao Liu, and Jiliang Tang. 2019. Say what I want: Towards the dark side of neural dialogue models. CoRR, abs/1909.06044. — [arXiv:1909.06044](https://arxiv.org/abs/1909.06044) find
  23. Haochen Liu, Wentao Wang, Yiqi Wang, Hui Liu, Zitao Liu, and Jiliang Tang. 2020a. Mitigating gender bias for neural dialogue generation with adversarial learning. arXiv preprint arXiv:2009.13028. find
  24. Haochen Liu, Zhiwei Wang, Tyler Derr, and Jiliang Tang. 2020b. Chat as expected: Learning to manipulate black-box neural dialogue models. arXiv preprint arXiv:2005.13170. — [arXiv:2005.13170](https://arxiv.org/abs/2005.13170) find
  25. Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871.
  26. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. CoRR, abs/1908.09635.
  27. Alexander H. Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. Parlai: A dialog research software platform. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September…
  28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  29. Marcelo O. R. Prates, Pedro H. C. Avelar, and Luı́s C. Lamb. 2018. Assessing gender bias in machine translation - A case study with google translate. CoRR, abs/1809.02208. find
  30. Alan Ritter, Colin Cherry, and William B. Dolan. 2011. Data-driven response generation in social media. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meeting of SIGDAT, a Speci…
  31. James A Rodger and Parag C Pendharkar. 2004. A field study of the impact of gender and user’s technical experience on the performance of voice-activated medical tracking application. International Journal of Human-Computer Studies, 60(5-6):529–544. find
  32. Adam Rose. 2010. Are face-detection cameras racist? Time Business. find
  33. Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankaranarayanan, and Sarath Chandar. 2018. Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intellige… find
  34. Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, pages 3776–3784.
  35. Iulian Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.
  36. Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natura…
  37. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. find
  38. Songül Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. 2019. Why machine learning may lead to unfairness: Evidence from risk assessment for juvenile justice in catalonia. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law, ICAIL 2019, Montreal, QC… find
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017,…
  40. Marty J. Wolf, Keith W. Miller, and Frances S. Grodzinsky. 2017. Why we should have seen that coming: comments on microsoft’s tay ”experiment, ” and wider implications. SIGCAS Computers and Society, 47(3):54– 64. find
  41. Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collaborative filtering. In Advances in Neural Information Processing Systems, pages 2921–2930. find
  42. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2015. Fairness constraints: Mechanisms for fair classification. find
  43. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. CoRR, abs/1804.06876.
  44. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 4847–4853.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2020-does-gender-matter-fairness-dialogue/