Elamur Research
☰
home / papers / 2022 / BBQ: A hand-built bias benchmark for question answering

BBQ: A hand-built bias benchmark for question answering

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2022.findings-acl.165.pdf

Summary

Language models are known to absorb social biases, but it is less clear how these biases appear in the concrete answers of applied systems such as question answering. Parrish et al. address this gap with BBQ, a benchmark of templates written by the authors. Each template targets a documented, harmful stereotype in one of nine social dimensions relevant to U.S. English-speaking settings, including age, disability, gender identity, race/ethnicity, religion and sexual orientation. Every item is a three-way multiple-choice question with an "unknown" option. Items come with either an ambiguous context, where admitting uncertainty is correct, or a disambiguated context that settles the answer. Questions are framed both negatively and non-negatively.

Evaluations of UnifiedQA, RoBERTa and DeBERTaV3 show two patterns. When information is missing, models often choose stereotype-consistent answers. When the context does contain the answer, they remain slightly less accurate if that answer contradicts a bias. BBQ matters because it measures bias in actual predictions rather than likelihoods, revealing in which situations and against which groups harm is likely.

Key points

  • The dataset has 325 crowdworker-validated templates, with at least 25 per category, expanding to over 58k examples (about 175 per template).
  • It is released under CC-BY 4.0 at github.com/nyu-mll/BBQ.
  • In ambiguous contexts, accuracy reaches at most 67.5%, and up to 77% of errors reinforce stereotypes.
  • In disambiguated contexts, accuracy is up to 3.4 points higher when the correct answer matches a bias, and the gap exceeds 5 points for gender for most models.
  • The authors caution that a low BBQ bias score does not prove a model is unbiased, especially outside the covered categories and U.S. context.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

310 papers of our library cite it (of the 140673 whose bibliographies we read); 24 of them have a page here, 286 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-acl.165 · arXiv 2110.08193

@inproceedings{parrish2022bbq,
  title = {{BBQ: A hand-built bias benchmark for question answering}},
  author = {Parrish, Alicia and Chen, Angelica and Nangia, Nikita and Padmakumar, Vishakh and Phang, Jason and Thompson, Jana and Htut, Phu Mon and Bowman, Samuel R.},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.165},
  eprint = {2110.08193},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.findings-acl.165}
}

References

39 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Bartz. 2009. English frst names for chinese americans. find
  2. Harvard University Social Science Statistics Blog. Accessed July 2021. find
  3. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of" bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476. find our card
  4. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint… find
  5. Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 7–15, Minneapolis, Minnesota. Associati… find our card
  6. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  7. Yang Trista Cao and Hal Daumé III. 2020. Toward gender-inclusive coreference resolution. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4568–4595, Online. Association for Computational Linguistics. find our card
  8. Jorida Cila, Richard N Lalonde, Joni Y Sasaki, Raymond A Mar, and Ronda F Lo. 2021. Zahra or Zoe, Arjun or Andrew? Bicultural baby names refect identity and pragmatic concerns. Cultural Diversity and Ethnic Minority Psychology, 27(3):307. find
  9. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457. find
  10. Kate Crawford. 2017. The trouble with bias. http: //youtube.com/watch?v=fMym\_BKWQzk. Talk given at NeurIPS December 2017. find
  11. Rajeev Darolia, Cory Koedel, Paco Martorell, Katie Wilson, and Francisco Perez-Arce. 2016. Race and gender effects on employer interest in job applicants: new evidence from a resume feld experiment. Applied Economics Letters, 23(12):853–856. find
  12. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Third Workshop on Abusive Language Online, pages 25–35, Florence, Italy. Association for Computational Linguistics. find our card
  13. Sunipa Dev, Emily Sheng, Jieyu Zhao, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Nanyun Peng, and Kai-Wei Chang. 2021. What do bias measures measure? arXiv preprint: 2108.03362. — [arXiv:2108.03362](https://arxiv.org/abs/2108.03362) find
  14. U.S. EEOC. 2021. Prohibited employment policies/practices. Accessed August 2021. find
  15. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543. find our card
  16. Joseph Kasof. 1993. Sex bias in the naming of stimulus persons. Psychological bulletin, 113(1):140. find
  17. Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020. UnifedQA: Crossing format boundaries with a single QA system. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 1896–1907. find
  18. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794, Copenhagen, Denmark. Association for Computational Ling… find our card
  19. Steven D Levitt and Stephen J Dubner. 2014. Freako-nomics. B DE BOOKS. find
  20. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Vivek Srikumar. 2020. UNQOVERing stereotyping biases via underspecifed questions. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3475–3489, Online. Association for Computational Linguistics. find
  21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. find
  22. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Aman-charla, and Anupam Datta. 2020. Gender bias in neural natural language processing. In Logic, Language, and Security, pages 189–202. Springer. find our card
  23. Liam Magee, Lida Ghahremanlou, Karen Soldatic, and Shanthi Robertson. 2021. Intersectional bias in causal language models. arXiv preprint arXiv:2107.07691. — [arXiv:2107.07691](https://arxiv.org/abs/2107.07691) find
  24. Victor Mair. 2018. Language log: Ask language log: Are east asian frst names gendered? Language Log. Accessed July 2021. find
  25. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and… find our card
  26. Robert Munro and Alex (Carmen) Morrison. 2020. Detecting independent pronoun bias with partially-synthetic data generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2011–2017, Online. Association for Computational Linguistics. find
  27. Keiko Nakao and Judith Treas. 1994. Updating occupational prestige and socioeconomic scores: How the new measures measure up. Sociological methodology, pages 1–72. find
  28. NYC OpenData. 2021. Popular baby names. Accessed July 2021. find
  29. Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Kalai. 2019. What’s in a name? Reducing bias in bios without access to protected attributes. In Proceedings of the 2019 Conference o… find
  30. Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference… find our card
  31. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 8–1… find our card
  32. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477–5490, Online. Association… find our card
  33. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process… find our card
  34. Konstantinos Tzioumis. 2018. Demographic aspects of frst names. Scientifc data, 5(1):1–9. find
  35. United States Census Bureau. 1990. Frequently occurring surnames from census 1990 – names fles. Accessed July 2021. find
  36. Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021. Learning from the worst: Dynamically generated datasets to improve online hate detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Nat… find our card
  37. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the gap: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–617. find our card
  38. Ellen Dionne Wu. 1999. “They call me Bruce, but they won’t call me Bruce Jones:” Asian American naming preferences and patterns. Names, 47(1):21–50. find
  39. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find our card