Self-Explaining Hate Speech Detection with Moral Rationales
Paper · 2026 · Francielle Vargas, Jackson Trager, Diego Alves et al.
Summary
Hate speech classifiers often depend on surface lexical cues. This makes them prone to spurious correlations, hard to interpret and weak with culturally contingent language. The authors argue that moral values, framed through Moral Foundations Theory, give a more stable signal than wording alone. They propose Supervised Moral Rationale Attention (SMRA), a self-explaining framework. During training, SMRA adds a loss that pushes the model's token-level attention toward spans that experts marked as moral rationales, so explanations come from the model itself rather than from post-hoc attribution.
To train and test SMRA, they release HateBRMoralXplain, the third version of the Brazilian Portuguese HateBR corpus. This version enriches the data with labels for moral foundations, word-level spans justifying those labels, and socio-political information about each comment.
On binary hate detection and multi-label moral classification, SMRA gives small predictive gains and clearly better agreement with human rationales, with no notable change in fairness. Tests with GPT-4o-mini and LLaMA-70B show that moral information in prompts helps hate detection, but LLMs remain weak at identifying moral categories. The work matters for building more transparent, culture-aware moderation tools, especially for lower-resource languages.
Key points
- HateBRMoralXplain corpus:
- 7,000 Instagram comments from public political accounts, balanced at 3,500 offensive and 3,500 non-offensive.
- Hate labels were annotated by three experts.
- Each comment receives one to three of the five MFT foundations (Care/Harm, Fairness/Cheating, Loyalty/Betrayal, Authority/Subversion, Purity/Degradation), ranked by salience, with rationales for each.
- Binary task results: SMRA raises Macro F1 by +0.0082 and improves rationale alignment, with IoU F1 up by +0.0743 and Token F1 up by +0.0503. Comprehensiveness falls by 0.0203 and Sufficiency by 0.0231.
- Multi-label moral task results: Macro F1 rises by +0.015 and Token F1 by +0.241. Fairness metrics stay roughly unchanged.
- LLM experiments (GPT-4o-mini, LLaMA-70B):
- Adding moral rationales to prompts improves hate detection by about 2–3%.
- Moral sentiment F1 stays below 0.40 for both Portuguese and English translations.
- LLM-generated rationales show a Jaccard similarity of only 0.11 with human rationales.
- Release and limitations: Data, per-annotator disagreements and code are available at github.com/franciellevargas/SMRA under CC BY-NC 4.0. The authors note that rationale annotation is expensive and that evaluation is so far limited to Brazilian Portuguese.
Institutions
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 works
🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksBias, fairness & safety · 517 works
Cite
Cite as Francielle Vargas, Jackson Trager, Diego Alves, Matteo Guida, Surendrabikram Thapa, Berk Atıl, Daryna Dementieva, Andrew J Smart and Ameeta Agrawal. 2026. Self-Explaining Hate Speech Detection with Moral Rationales. Findings of the Association for Computational Linguistics: ACL 2026.
@inproceedings{vargas2026self,
title = {{Self-Explaining Hate Speech Detection with Moral Rationales}},
author = {Francielle Vargas, Jackson Trager, Diego Alves, Matteo Guida, Surendrabikram Thapa, Berk Atıl, Daryna Dementieva, Andrew J Smart and Ameeta Agrawal},
booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
year = {2026},
doi = {10.18653/v1/2026.findings-acl.1704},
url = {https://doi.org/10.18653/v1/2026.findings-acl.1704}
}References
- Suhaib Abdurahman, Mohammad Atari, Farzan Karimi-Malekabadi, Mona J Xue, Jackson Trager, Peter S Park, Preni Golazizian, Ali Omrani, and Morteza Dehghani. 2024. Perils and opportunities in using large language models in psychological research. PNAS nexus, 3(7):pgae245. find
- Gavin Abercrombie, Dirk Hovy, and Vinodkumar Prabhakaran. 2023. Temporal and second language influence on intra-annotator agreement and stability in hate speech labelling. In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pages 96–103, Toronto, Canada. Association for Computation… find
- Mohammad Atari, Jonathan Haidt, Jesse Graham, Sena Koleva, Sean T Stevens, and Morteza Dehghani. 2023. Morality beyond the weird: How the nomological network of morality varies across cultures. Journal of Personality and Social Psychology, 125(5):1157. find
- Giuseppe Attanasio, Debora Nozza, Dirk Hovy, and Elena Baralis. 2022. c. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1105–1119, Dublin, Ireland. Association for Computational Linguistics. find
- Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Held Online.
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion Proceedings of The 2019 World Wide Web Conference, WWW ’19, page 491–500, New York, NY, USA. Association for Comput… find
- William J Brady, Killian L McLoughlin, Mark P Torres, Kara F Luo, Maria Gendron, and MJ Crockett. 2023. Overperception of moral outrage in online social networks inflates beliefs about intergroup hostility. Nature human behaviour, 7(6):917–927. find
- William J Brady, Julian A Wills, John T Jost, Joshua A Tucker, and Jay J Van Bavel. 2017. Emotion shapes the diffusion of moralized content in social networks. Proceedings of the National Academy of Sciences, 114(28):7313–7318. find
- Agostina Calabrese, Björn Ross, and Mirella Lapata. 2022. Explainable abuse detection as intent classification and slot filling. Transactions of the Association for Computational Linguistics, 10:1440–1454. find
- Amanda Cercas Curry, Giuseppe Attanasio, Debora Nozza, and Dirk Hovy. 2023. MilaNLP at SemEval-2023 task 10: Ensembling domain-adapted and regularized pretrained language models for robust sexism detection. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), page… find
- Aida Davani, Mark Díaz, Dylan Baker, and Vinodkumar Prabhakaran. 2024a. D3CODE: Disentangling disagreements in data across cultures on offensiveness detection and evaluation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18511–18526, Miami, Florida,… find
- Aida Davani, Mark Díaz, Dylan Baker, and Vinodkumar Prabhakaran. 2024b. Disentangling perceptions of offensiveness: Cultural and moral correlates. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2007–2021, Rio de Janeiro, Brazil. find
- Aida Mostafazadeh Davani, Mohammad Atari, Brendan Kennedy, and Morteza Dehghani. 2023. Hate speech classifiers learn normative social stereotypes. Transactions of the Association for Computational Linguistics, 11:300–319.
- Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the 3rd Workshop on Abusive Language Online, pages 25–35, Florence, Italy.
- Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443–4458, Online. Asso…
- Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’18, page 67–73, New York, NY, USA. Association for Computing Machinery. find
- Brage Eilertsen, Røskva Bjørgfinsdóttir, Francielle Vargas, and Ali Ramezani-Kebrya. 2025. Aligning attention with human rationales for self-explaining hate speech detection. In Proceedings of the 40th AAAI Conference on Artificial Intelligence. Alignment Track., pages 1–15, Singapore, Singapore. find
- Jim AC Everett. 2013. The 12 item social and economic conservatism scale (secs). PloS one, 8(12):e82131. find
- Paula Fortuna, João Rocha da Silva, Juan Soler-Company, Leo Wanner, and Sérgio Nunes. 2019. A hierarchically-labeled Portuguese hate speech dataset. In Proceedings of the Third Workshop on Abusive Language Online, pages 94–104, Florence, Italy. Association for Computational Linguistics.
- Michele J Gelfand, Jana L Raver, Lisa Nishii, Lisa M Leslie, Janetta Lun, Beng Chong Lim, Lili Duan, Assaf Almaliach, Soon Ang, Jakobina Arnadottir, and 1 others. 2011. Differences between tight and loose cultures: A 33-nation study. science, 332(6033):1100–1104. find
- Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in experimental social psychology, volume 47, pages 55–130. Elsevier. find
- Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
- Rongchen Guo, Isar Nejadgholi, Hillary Dawkins, Kathleen C Fraser, and Svetlana Kiritchenko. 2024. Adaptable moral stances of large language models on sexist content: Implications for society and gender discourse. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Process… find
- Kobi Hackenburg, William J Brady, and Manos Tsakiris. 2023. Mapping moral language on us presidential primary campaigns reveals rhetorical networks of political division and unity. PNAS nexus, 2(6):pgad189. find
- Arnold K Ho, Jim Sidanius, Nour Kteily, Jennifer Sheehy-Skeffington, Felicia Pratto, Kristin E Henkel, Rob Foels, and Andrew L Stewart. 2015. The nature of social dominance orientation: Theorizing and measuring preferences for intergroup inequality using the new sdo7 scale. Journal of personality an… find
- Paul Jaccard. 1901. Étude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin de la Société Vaudoise des Sciences Naturelles, 37:547–579. find
- Peter K Jonason and Gregory D Webster. 2010. The dirty dozen: a concise measure of the dark triad. Psychological assessment, 22(2):420. find
- Farzan Karimi-Malekabadi, Suhaib Abdurahman, Zhivar Sourati, Jackson Trager, and Morteza Dehghani. 2026. Theory trace card for llm socio-cognitive evaluation. Manuscript. Preprint. find
- Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Joe Hoover, Ali Omrani, Jesse Graham, and Morteza Dehghani. 2021. Moral concerns are differentially observable in language. Cognition, 212:104696. find
- Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. find
- Jari-Mikko Meriläinen. 2024. The role of gender in hate speech targeting politicians: Evidence from finnish twitter. International Journal of Politics, Culture, and Society, pages 1–27. find
- Brendan Kennedy, Preni Golazizian, Jackson Trager, Mohammad Atari, Joe Hoover, Aida Mostafazadeh Davani, and Morteza Dehghani. 2023. The (moral) language of hate. PNAS nexus, 2(7):pgad210. find
- Anirudh Mittal, Pranav Jeevan P, Prerak Gandhi, Diptesh Kanojia, and Pushpak Bhattacharyya. 2021. “so you think you’re funny?”: Rating the humour quotient in standup comedy. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10073–10079, Online and Punta… find
- Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020. Contextualizing hate speech classifiers with post-hoc explanation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5435–5442, Online. Association for Computa…
- Ayushi Nirmal, Amrita Bhattacharjee, Paras Sheth, and Huan Liu. 2024. Towards interpretable hate speech detection using large language model-extracted rationales. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 223–233, Mexico City, Mexico. Association for Computation… find
- Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024. CLIcK: A benchmark dataset of cultural and linguistic intelligence in Korean. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024… find
- Jiyun Kim, Byounghan Lee, and Kyung-Ah Sohn. 2022. Why is it hate speech? masked rationale prediction for explainable hate speech detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6644–6655, Gyeongju, Republic of Korea. International Committee on Comp… find
- Nayeon Lee, Chani Jung, and Alice Oh. 2023. Hate speech classifiers are culturally insensitive. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 35–46, Dubrovnik, Croatia. Association for Computational Linguistics. find
- Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
- Marian Marchal, Merel Scholman, Frances Yung, and Vera Demberg. 2022. Establishing annotation quality in multi-label annotations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3659–3668, Gyeongju, Republic of Korea. International Committee on Computational L… find
- Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, volume 35, pages 14867–14875, Held Online. find
- OpenAI. 2024. Gpt-4o mini: advancing cost-efficient intelligence. find
- Niels Bjørn Grund Petersen, Rasmus Tue Pedersen, and Mads Thau. 2025. Citizens’ perceptions of online abuse directed at politicians: Evidence from a survey experiment. European Journal of Political Research, 64(2):790–809. find
- Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55(3):477–523. find
- Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, and Mark Diaz. 2021. On releasing annotator-level labels and information in datasets. In Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop, pages 133–138, Punta Cana, Dominican…
- Rezvaneh Rezapour, Ly Dinh, and Jana Diesner. 2021. Incorporating the measurement of moral foundations theory into analyzing stances on controversial topics. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media, page 177–188, New York, NY, USA. Association for Computing Machinery. find
- Ludovic Rheault, Erica Rayment, and Andreea Musulan. 2019. Politicians in the line of fire: Incivility and the treatment of women on social media. Research & Politics, 6(1):2053168018816228. find
- Isadora Salles, Francielle Vargas, and Fabrício Benevenuto. 2025a. HateBRXplain: A benchmark dataset with human-annotated rationales for explainable hate speech detection in Brazilian Portuguese. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6659–6669, Abu D… find
- Jackson Trager and Mohammad Atari. 2025. The immorality of too much money. PNAS nexus, 4(6):pgaf158. find
- Isadora Salles, Francielle Vargas, and Fabrício Benevenuto. 2025b. Hatebrxplain: A benchmark dataset with human-annotated rationales for explainable hate speech detection in brazilian portuguese. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6659–6669. find
- Jackson Trager, Farzan Karimi-Malekabadi, Suhaib Abdurahman, and Morteza Dehghani. 2025a. Hate is justified when values are threatened: Evidence from ideologically threatening tweets and real world events. arxiv. find
- Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678, Florence, Italy.
- Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Ling…
- Kristina Bakkær Simonsen and Tobias Widmann. 2025. When do political parties moralize?: A cross-national study of the use of moral language in political communication on immigration. British Journal of Political Science, 55:e33. find
- Theodore M Singelis, Harry C Triandis, Dharm PS Bhawuk, and Michele J Gelfand. 1995. Horizontal and vertical dimensions of individualism and collectivism: A theoretical and measurement refinement. Cross-cultural research, 29(3):240–275. find
- Kirill Solovev and Nicolas Pröllochs. 2023. Moralized language predicts hate speech on social media. PNAS nexus, 2(1):pgac281. find
- Kirill Solovev and Nicolas Pröllochs. 2022. Moralized language predicts hate speech on social media. PNAS Nexus, 2(1):pgac281. find
- Christopher J Soto and Oliver P John. 2017. Short and extra-short forms of the big five inventory–2: The bfi-2-s and bfi-2-xs. Journal of Research in Personality, 68:69–81. find
- Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020. BERTimbau: pretrained BERT models for Brazilian Portuguese. In 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23 (to appear). find
- Manuel Tonneau, Diyi Liu, Samuel Fraiberger, Ralph Schroeder, Scott A. Hale, and Paul Röttger. 2024. From languages to geographies: Towards evaluating cultural bias in hate speech datasets. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 283–311, Mexico City, Mexico.… find
- Jackson Trager, Francielle Vargas, Diego Alves, Matteo Guida, Mikel K Ngueajio, Ameeta Agrawal, Yalda Daryani, Farzan Karimi-Malekabadi, and Flor Miriam Plaza-del Arco. 2025b. Mftcxplain: A multilingual benchmark dataset for evaluating the moral reasoning of llms through multi-hop hate speech explan… find
- Jackson Trager, Alireza S Ziabari, Aida Mostafazadeh Davani, Preni Golazizian, Farzan Karimi-Malekabadi, Ali Omrani, Zhihe Li, Brendan Kennedy, Nils Karl Reimer, Melissa Reyes, and 1 others. 2022. The moral foundations reddit corpus. arXiv preprint arXiv:2208.05545. — [arXiv:2208.05545](https://arxiv.org/abs/2208.05545) find
- Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023. Toxicity, morality, and speech act guided stance detection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4464–4478, Singapore. Association for Computational Linguistics. find
- Francielle Vargas, Isabelle Carvalho, Fabiana Rodrigues de Góes, Thiago Pardo, and Fabrício Benevenuto. 2022a. Hatebr: A large expert annotated corpus of brazilian instagram comments for offensive language and hate speech detection. In Proceedings of the Thirteenth Language Resources and Evaluation… find
- Francielle Vargas, Isabelle Carvalho, Ali Hürriyetoğlu, Thiago Pardo, and Fabrício Benevenuto. 2023. Socially responsible hate speech detection: Can classifiers reflect social stereotypes? In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 1… find
- Francielle Vargas, Isabelle Carvalho, Thiago A. S. Pardo, and Fabrício Benevenuto. 2024. Context-aware and expert data resources for brazilian portuguese hate speech detection. Natural Language Processing, 31(2):435–456. find
- Francielle Vargas, Isabelle Carvalho, Fabiana Rodrigues de Góes, Thiago Pardo, and Fabrício Benevenuto. 2022b. HateBR: A large expert annotated corpus of Brazilian Instagram comments for offensive language and hate speech detection. In Proceedings of the Thirteenth Language Resources and Evaluation… find
- Francielle Vargas, Fabiana Rodrigues de Góes, Isabelle Carvalho, Fabrício Benevenuto, and Thiago Pardo. 2021. Contextual-lexicon approach for abusive language detection. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1438–1447, He… find
- Sze-Yuh Nina Wang and Yoel Inbar. 2021. Moral-language use by us political elites. Psychological Science, 32(1):14–26. find
- Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 602–608, Minneapoli…
- Sidney Wong. 2024. Sociocultural considerations in monitoring anti-LGBTQ+ content on social media. In Proceedings of the 2nd Workshop on Cross-Cultural Considerations in NLP, pages 84–97, Bangkok, Thailand. Association for Computational Linguistics. find
- Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using “annotator rationales” to improve machine learning for text categorization. In Human language technologies 2007: The conference of the North American chapter of the association for computational linguistics; proceedings of the main confere…
- Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human…
- Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
- Li Zhou, Antonia Karamolegkou, Wenyu Chen, and Daniel Hershcovich. 2023. Cultural compass: Predicting transfer learning success in offensive language detection with cultural features. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12684–12702, Singapore. Association… find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/vargas-2026-self-explaining-hate-speech-detection/