Elamur Research⌕
☰
home / papers / 2020 / Towards Controllable Biases in Language Generation

Towards Controllable Biases in Language Generation

Paper · 2020 · Emily Sheng, Kai-Wei Chang, Prem Natarajan et al.

Summary

The paper addresses societal bias in natural language generation, where model outputs can shape how people perceive demographic groups. The authors extend the gradient-based adversarial trigger search of Wallace et al. (2019) into a model-agnostic framework. A short, usually nonsensical token sequence is optimized so that, when placed before prompts mentioning a demographic group (gender, race, sexual orientation), it pushes generated text toward chosen regard polarities. The objective combines association with desired target samples and dissociation from undesired ones. Bias is measured with the regard metric of Sheng et al. (2019).

The authors use this framework in two ways. First, they induce opposite biases for paired groups to analyze what GPT-2 has learned. Second, they find a trigger that equalizes regard across groups. They then apply the mitigation trigger to DialoGPT. In both models, the trigger narrows the regard gap between groups and lowers the share of negative outputs overall. This offers a practical way to audit and reduce bias in deployed generators without retraining them.

Key points

  • The framework is evaluated on GPT-2 and on the dialogue system DialoGPT. Bias is measured with regard, a measure of social perception that can differ from sentiment.
  • Without mitigation, DialoGPT responses mentioning Black names averaged 0.30 regard, compared with 0.37 for White names (2,000 samples each). A random-partition test gave a z-score of 22.7 and p ≈ 1.7 × 10⁻¹¹⁴.
  • With the mitigation trigger, responses with Black names reached an average regard of 0.53, and those with White names reached 0.52. The trigger was found using 16 names, and the effect also held for unseen names.
  • In the bias analysis, a trigger that disadvantages RACE-BLACK relative to RACE-WHITE led to more generated text about international relations. Inducing negative bias against White was relatively harder than against Black, compared with the straight vs. gay pair.
  • Human annotation of 200 samples showed an inter-annotator correlation of 0.71 and a correlation of 0.66 with automatic labels. Code and data are available at github.com/ewsheng/controllable-nlg-biases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Southern CaliforniaLos Angeles · United States104 works · 33,876 citations🎓University of California, Los AngelesLos Angeles · United States54 works · 53,688 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSocial Sciences · 744 worksSafety Research · 221 works

🧭 Topics 1Research subjects and languages the work is about.Text generation · 145 works

Cite

Cite as Emily Sheng, Kai-Wei Chang, Prem Natarajan and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. Findings of the Association for Computational Linguistics: EMNLP 2020.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.findings-emnlp.291

@inproceedings{sheng2020controllable,
  title = {{Towards Controllable Biases in Language Generation}},
  author = {Emily Sheng, Kai-Wei Chang, Prem Natarajan and Nanyun Peng},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.291},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.291}
}

References

27 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349–4357.
  2. Shikha Bordia and Samuel Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 7–15.
  3. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations.
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Jessica Ficler and Yoav Goldberg. 2017. Controlling linguistic style aspects in neural language generation. In Proceedings of the Workshop on Stylistic Variation, pages 94–104.
  6. Liye Fu, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Tie-breaker: Using language models to quantify gender bias in sports journalism. In Proceedings of the IJCAI workshop on NLP meets Journalism. find
  7. Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight. 2017. Hafez: an interactive poetry generation system. In Proceedings of ACL 2017, System Demonstrations, pages 43–48.
  8. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. volume 70 of Proceedings of Machine Learning Research, pages 1587–1596, International Convention Centre, Sydney, Australia. PMLR.
  9. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858.
  10. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166–172.
  11. S. D. Levitt and S. J. Dubner. 2005. Freakonomics: A rogue economist explores the hidden side of everything. William Morrow and Co. find
  12. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018. Gender bias in neural natural language processing. arXiv preprint arXiv:1807.11714.
  13. Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456.
  14. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799–2804.
  15. Vinodkumar Prabhakaran, Ben Hutchinson, and Margaret Mitchell. 2019. Perturbation sensitivity analysis to detect unintended model biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin… find
  16. Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically neutralizing subjective bias in text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 480–489. find
  17. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  18. Language Technologies, Volume 2 (Short Papers), pages 8–14. find
  19. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725.
  20. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (…
  21. Vered Shwartz, Rachel Rudinger, and Oyvind Tafjord. 2020. " you are grounded!": Latent name artifacts in pre-trained language models. arXiv preprint arXiv:2004.03012. find
  22. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find
  23. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration.
  24. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1…
  25. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  26. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853.
  27. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sheng-2020-controllable-biases-language-generation/