Elamur Research
☰
home / papers / 2020 / Social Bias Frames: Reasoning about Social and Power Implications of Language

Social Bias Frames: Reasoning about Social and Power Implications of Language

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.486.pdf

Summary

Toxicity detectors usually give a post one label, such as offensive or not. A label like that doesn't say why a post is harmful, and harm often comes from what a post implies rather than what it says. The authors propose Social Bias Frames, a formalism for the social bias a post projects. It mixes categorical judgments with free-text explanations: whether the post is offensive, whether it was meant to offend, whether it is lewd, whether it targets a group or only an individual, which group it targets, what stereotype it implies, and whether the speaker might belong to that group. To support the formalism, they crowdsourced the Social Bias Inference Corpus (SBIC). It covers posts from Reddit, Twitter and hate sites such as Gab and Stormfront. They then trained baseline models built on pretrained language models, including GPT-2 variants. The models can decide fairly well whether a post is offensive, at about 80% F1. They do poorly at writing the implied stereotype and tend to produce generic statements about a group. The work moves bias detection toward explainable reasoning that people can check.

Key points

  • SBIC has about 150k structured annotations on 44,671 posts. They include over 34k implied statements about roughly a thousand demographic groups (the related-work section gives about 1.4k).
  • Each frame has seven variables. Offensiveness, intent, lewdness, group implication and in-group language are categorical. The targeted group and the implied statement are free text, with implications written as simple Hearst-style patterns (for example, "GROUP does ___").
  • The posts come from r/darkJokes, r/meanJokes, r/offensiveJokes and Microaggressions on Reddit, from existing Twitter toxicity datasets (Founta et al., Davidson et al., Waseem and Hovy), and from Gab, Stormfront and banned subreddits (r/Incels, r/MensRights).
  • Error analysis of SBF-GPT2-gdy-constr shows it usually gets the stereotype right only when the stereotype shares many words with the post.
  • The authors frame bias using a U.S. sociocultural lens. To protect annotators, they capped how many posts each person could label per day, paid $7–12, and pointed workers to crisis-support resources.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

348 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 326 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.486 · arXiv 1911.03891

@inproceedings{sap2020social,
  title = {{Social Bias Frames: Reasoning about Social and Power Implications of Language}},
  author = {Sap, Maarten and Gabriel, Saadia and Qin, Lianhui and Jurafsky, Dan and Smith, Noah A. and Choi, Yejin},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.486},
  eprint = {1911.03891},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.486}
}

References

75 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maria Antoniak, David Mimno, and Karen Levy. 2019. Narrative paths and negotiation of power in birth stories. In CSCW. find
  2. John W Ayers, Theodore L Caputi, Camille Nebeker, and Mark Dredze. 2018. Don’t quote me: reverse identification of research participants in social media studies. NPJ digital medicine, 1(1):1–2. find
  3. Maria Becker, Michael Staniek, Vivi Nastase, and Anette Frank. 2017. Enriching argumentative texts with implicit knowledge. In NLDB. find
  4. Jeanette Bicknell. 2007. What is offensive about offensive jokes? Philosophy Today, 51(4):458–465. find
  5. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic dialectal variation in social media: a case study of African-American English. In EMNLP. find our card
  6. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: commonsense transformers for automatic knowledge graph construction. In ACL. find our card
  7. Lorraine Bowman-Grieve. 2009. Exploring “Stormfront”: a virtual community of the radical right. Studies in conflict & terrorism, 32(11):989–1007. find
  8. Luke M Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019. Finding microaggressions in the wild: a case for locating elusive phenomena in social media posts. In EMNLP. find
  9. Adrian Bussone, Simone Stumpf, and Dympna O’Sullivan. 2015. The role of explanations on trust and reliance in clinical decision support systems. In 2015 International Conference on Healthcare Informatics, pages 160–169. IEEE. find
  10. Southern Poverty Law Center. 2012. Misogyny: the sites. Intelligence Report, 145. find
  11. Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN -COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech. In ACL. find our card
  12. Raphael Cohen-Almagor. 2014. Countering hate on the internet. Annual review of law and ethics, 22:431–443. find
  13. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In KDD. find
  14. Adam M Croom. 2011. Slurs. Language Sciences, 33(3):343–358. find
  15. Helen L Daly. 2018. On insults. Journal of the American Philosophical Association, 4(4):510–524. find
  16. Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. 2012. Echoes of power: language effects and power differences in social interaction. In WWW. find
  17. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Abusive Language Workshop. find our card
  18. Thomas Davidson, Dana Warmsley, Michael W Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In ICWSM. find our card
  19. Serena Does, Belle Derks, and Naomi Ellemers. 2011. Thou shalt not discriminate: how emphasizing moral ideals rather than obligations increases whites’ support for social equality. Journal of Experimental Social Psychology, 47(3):562–571. find
  20. Marta Dynel. 2015. The landscape of impoliteness research. Journal of Politeness Research, 11(2):383. find
  21. Connie C Eble. 1996. Slang & sociability: in-group language among college students. Univ of North Carolina Press. find
  22. Anjalie Field, Gayatri Bhat, and Yulia Tsvetkov. 2019. Contextual affective analysis: a case study of people portrayals in online \#MeToo stories. In ICWSM. find
  23. Charles J Fillmore and Collin F Baker. 2001. Frame semantics for text understanding. In Proceedings of WordNet and Other Lexical Resources Workshop, NAACL. find
  24. Jon Fingas. 2017. Reddit bans misogynist community as part of anti-violence crackdown. https: //www.engadget.com/2017/11/08/ reddit-bans-misogynist-community-in-anti-violence-crackdown/. Accessed: 2019-12-06. find
  25. Susan T Fiske. 1993. Controlling other people. the impact of power on stereotyping. American psychologist, 48(6):621–628. find
  26. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of Twitter abusive behavior. In ICWSM. find
  27. Yiannis Gabriel. 1998. An introduction to the social psychology of insults in organizations. Human Relations, 51(11):1329–1354. find
  28. Adam D Galinsky, Cynthia S Wang, Jennifer A Whitson, Eric M Anicich, Kurt Hugenberg, and Galen V Bodenhausen. 2013. The reappropriation of stigmatizing labels: the reciprocal relationship between power and self-labeling. Psychol. Sci., 24(10):2020–2029. find
  29. Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, and William B. Dolan. 2015. deltaBLEU: a discriminative metric for generation tasks with intrinsically diverse targets. In ACL. find
  30. Ona de Gibert, Naiara Pérez, Aitor Garcı́a-Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. In Abusive Language Workshop at EMNLP. find our card
  31. Gil Greengross and Geoffrey F Miller. 2008. Dissing oneself versus dissing rivals: effects of status, personality, and sex on the Short-Term and Long-Term attractiveness of Self-Deprecating and Other-Deprecating humor. Evolutionary Psychology, 6(3). find
  32. Shirley Gregor and Izak Benbasat. 1999. Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS quarterly, pages 497–530. find
  33. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? empirical analysis and detecting attributes of convincingness in web argumentation. In EMNLP, pages 1214–1223. find
  34. Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. 2019. Unifying human and statistical evaluation for natural language generation. In NAACL-HLT. find
  35. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In ACL, pages 539– 545. find
  36. Amanda Hess. 2016. The far right has a new digital safe space. https://www.nytimes. com/2016/11/30/arts/the-far-right-has-a-new-digital-safe-space.html. Accessed: 2019-12-06. ↗
  37. Gabriele Kasper. 1990. Linguistic politeness: current research issues. Journal of Pragmatics, 14(2):193– 218. find
  38. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: a conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. find our card
  39. Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more? The effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1–10. ACM. find
  40. Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. 2015. From word embeddings to document distances. In ICML, pages 957–966. find
  41. Robin Lakoff. 1973. Language and woman’s place. Language in society, 2(1):45–79. find
  42. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In ACL. find
  43. Marie-Catherine de Marneffe, Christopher D Manning, and Christopher Potts. 2012. Did it happen? the pragmatic complexity of veridicality assessment. Computational Linguistics, 38(2):301–333. find our card
  44. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In FAccT. find
  45. An T Nguyen, Aditya Kharosekar, Saumyaa Krishnan, Siddhesh Krishnan, Elizabeth Tate, Byron C Wallace, and Matthew Lease. 2018. Believe it or not: designing a human-AI partnership for mixed-initiative fact-checking. In The 31st Annual ACM Symposium on User Interface Software and Technology, pages 189… find
  46. Conor J O’Dea, Stuart S Miller, Emma B Andres, Madelyn H Ray, Derrick F Till, and Donald A Saucier. 2015. Out of bounds: Factors affecting the perceived offensiveness of racial slurs. Language Sciences, 52:155–164. find
  47. Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019. Multilingual and Multi-Aspect hate speech analysis. In EMNLP. find our card
  48. Gonçalo Pereira, Rui Prada, and Pedro A Santos. 2016. Integrating social power into the decision-making of cognitive agents. Artificial Intelligence, 241:1–44. find
  49. Vinodkumar Prabhakaran, Prabhakaran Vinodkumar, and Rambow Owen. 2014. Predicting power relations between participants in written dialog from a single thread. In ACL. find
  50. Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In EMNLP. find
  51. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Unpublished. find
  52. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Unpublished. find
  53. Hannah Rashkin, Maarten Sap, Emily Allaway, Noah A. Smith, and Yejin Choi. 2018. Event2mind: commonsense inference on events, intents, and reactions. In ACL. find
  54. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?”: Explaining the predictions of any classifier. In KDD. find our card
  55. Sarah T Roberts. 2016. Commercial content moderation: digital laborers’ dirty work. In Safiya Umoja Noble and Brendesha M Tynes, editors, The Intersectional Internet: Race, Sex, Class and Culture Online, Media Studies Publications. Peter Lang Publishing. find
  56. Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. 2017. Measuring the reliability of hate speech annotations: the case of the european refugee crisis. In NLP 4 CMC Workshop. find
  57. RWJF. 2017. Discrimination in america: experiences and views. https://www.rwjf. org/en/library/research/2017/ 10/discrimination-in-america-- experiences-and-views.html. Accessed: 2019-11-5. ↗
  58. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: an adversarial winograd schema challenge at scale. In AAAI. find
  59. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019a. The risk of racial bias in hate speech detection. In ACL. find our card
  60. Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019b. ATOMIC: an atlas of machine commonsense for if-then reasoning. In AAAI. find
  61. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In NAACL. find our card
  62. Michael Zimmer. 2018. Addressing conceptual gaps in big data research ethics: an application of contextual integrity. Social Media + Society, 4(2). find
  63. Maarten Sap, Marcella Cindy Prasetio, Ariel Holtzman, Hannah Rashkin, and Yejin Choi. 2017. Connotation frames of power and agency in modern films. In EMNLP. find
  64. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019c. Social IQa: commonsense reasoning about social interactions. In EMNLP. find
  65. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Workshop on NLP for Social Media at EACL. find our card
  66. Robyn Speer and Catherine Havasi. 2012. Representing general relational knowledge in ConceptNet 5. In LREC. find
  67. Whitney Strub. 2008. The clearly obscene and the queerly obscene: heteronormativity and obscenity in cold war los angeles. American Quarterly, 60(2):373–398. find
  68. Stefanie Ullmann and Marcus Tomalin. 2019. Quaran-tining online hate speech: technical and ethical perspectives. Ethics and Information Technology. find
  69. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. find
  70. James Vincent. 2016. Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day. https://www.theverge.com/2016/3/ 24/11297050/tay-microsoft-chatbot-racist. Accessed: 2019-10-26. ↗
  71. Andrew J Vonasch and Roy F Baumeister. 2017. Unjustified side effects were strongly intended: taboo tradeoffs and the side-effect effect. Journal of Experimental Social Psychology, 68:83–92. find
  72. Zijian Wang and Christopher Potts. 2019. TalkDown: a corpus for condescension detection in context. In EMNLP. find
  73. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In NAACL Student Research Workshop. find
  74. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: personal attacks seen at scale. In WWW. model offensive 42.2% pos. (dev.) F 1 pr. rec. intent 44.8% pos. (dev.) F 1 pr. rec. lewd 3.0% pos. (dev.) F 1 pr. rec. group 66.6% pos. (dev.) F 1 pr. rec. in-group 5.1% pos. (dev.) F 1 pr. rec.… find
  75. SBF-GPT 1 -gdy 00 -constr SBF-GPT 2 -gdy 00 -constr SBF-GPT 2 -smp 00 -constr 75.2 88.3 75.2 88.3 77.2 88.3 77.2 88.3 80.5 84.3 80.4 84.3 74.4 89.8 74.4 89.8 76.3 89.5 76.3 89.5 75.3 89.9 75.3 89.9 63.6 63.6 66.5 66.5 64.7 64.7 75.2 78.2 72.5 75.2 78.2 72.5 77.6 81.2 74.3 77.6 81.2 74.3 78.6 80.6 76… find