Elamur Research⌕
☰
home / papers / 2026 / SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context

SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context

Paper · 2026 · Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas et al.

Summary

Benchmarks for testing whether generative models reproduce stereotypes are skewed toward Western contexts. Sub-Saharan Africa has almost no locally sourced stereotype data. The authors built SAFARI, a multilingual resource for Ghana, Kenya, Nigeria and South Africa, and designed a collection method suited to the region. Written or web-based surveys face obstacles there, including costly connectivity, missing keyboard support for many orthographies and strong oral traditions. The team therefore worked with the local organizations Mantaray and RAIN Africa to run phone interviews moderated in each participant's preferred language. Recruitment used ethnicity-based quotas. Responses were mostly written down in English first, and professional linguists then rendered them into the native languages. Each entry pairs an identity term with an attribute and an identity axis, plus the participant's own ratings of offensiveness and prevalence. The result covers thousands of stereotypes in English and 15 local languages. It offers evaluation data for model safety in an underserved region and a reusable template for participatory, oral-first data collection.

Key points

  • The data come from 410 participants, at least 100 per country, recruited through controlled quota sampling across major ethnic groups. It contains 3,534 English stereotypes and 3,206 stereotypes in 15 native languages, such as Kiswahili (954 entries), Twi, Hausa, Yoruba, Zulu and Afrikaans.
  • 1,919 entries target ethnic groups and 382 concern specific regions or states, so the coverage is fine-grained and local.
  • Participants rated each stereotype for offensiveness on a 1–5 scale and for prevalence on a 1–4 scale. Each interview yielded 10–15 stereotypes, with no more than three per identity term.
  • Only Swahili was transcribed directly. All other languages went through English and were then translated back by linguists. Moderators noted that this route can weaken how offensive a stereotype sounds.
  • The dataset is released at github.com/google-research-datasets/SAFARI and is intended for model evaluation and bias mitigation. It is not meant to certify that a model is free of bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 517 works

Cite

Cite as Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas, Andrew Zaldivar, Vinodkumar Prabhakaran and Sunipa Dev. 2026. SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-short.27

@inproceedings{verma2026safari,
  title = {{SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context}},
  author = {Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas, Andrew Zaldivar, Vinodkumar Prabhakaran and Sunipa Dev},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-short.27},
  url = {https://doi.org/10.18653/v1/2026.eacl-short.27}
}

References

49 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Abejide Ade-Ibijola and Chinedu Okonkwo. 2023. Artificial intelligence in africa: Emerging challenges. In Responsible AI in Africa: Challenges and opportunities, pages 101–117. Springer International Publishing Cham. find
  3. Ife Adebara and Muhammad Abdul-Mageed. 2022. Towards afrocentric NLP for African languages: Where we are and where we can go. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3814–3841, Dublin, Ireland. Association for Computat… find
  4. David Ifeoluwa Adelani, Jesujoba Oluwadara Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen H. Muhammad, Guyo D.… — A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
  5. Ibrahim Said Ahmad, Shiran Dudy, Resmi Ramachan-dranpillai, and Kenneth Church. 2024. Are generative language models multicultural? a study on hausa culture and emotions using chatgpt. Preprint, arXiv:2406.19504. find
  6. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, and 1 others. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403.
  7. Anthropic. The claude 3 model family: Opus, sonnet, haiku. find
  8. Samuel Opoku Appiah and Alfredo Ardila. 2021. The dilemma of instructional language in education: The case of ghana. Hungarian Educational Research Journal, 11(4):440–448. find
  9. Ayo̧ Bamgboşe. 2000. Language and Exclusion: The Consequences of Language Policies in Africa. Contributions to Studies on African Languages and Literatures. Lit Verlag, Münster ; London. find
  10. Mukul Bhutani, Kevin Robinson, Vinodkumar Prabhakaran, Shachi Dave, and Sunipa Dev. 2024. SeeG-ULL multilingual: a dataset of geo-culturally situated stereotypes. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 842–854, Bangk… find
  11. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. Preprint, arXiv:2110.01963. — [arXiv:2110.01963](https://arxiv.org/abs/2110.01963) find
  12. Aaron Chatterji, Tom Cunningham, David Deming, Zoë Hitzig, Zoe Hitzig, Christopher Ong, Carl Shan, and Kevin Wadman. 2025. How people use chatgpt. NBER Working Paper. find
  13. Thokozani Chimkono, Eunice Mphako-Banda, Amelia Taylor, and Pascal Kishindo. 2021. A usability study of the central-bantu multilingual keyboards. International Journal of Human–Computer Interaction, 37(16):1489–1503. find
  14. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3290 other…
  15. Aida Mostafazadeh Davani, Mohammad Atari, Brendan Kennedy, and Morteza Dehghani. 2023. Hate speech classifiers learn normative social stereotypes. Transactions of the Association for Computational Linguistics, 11:300–319.
  16. Aida Mostafazadeh Davani, Sunipa Dev, Héctor Pérez-Urbina, and Vinodkumar Prabhakaran. 2025. A comprehensive framework to operationalize social stereotypes for responsible AI evaluations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 30030–30043, Su… find
  17. Sunipa Dev, Jaya Goyal, Dinesh Tewari, Shachi Dave, and Vinodkumar Prabhakaran. 2023. Building socio-culturally inclusive stereotype resources with community engagement. Preprint, arXiv:2307.10514. — [arXiv:2307.10514](https://arxiv.org/abs/2307.10514) find
  18. Sunipa Dev, Tao Li, Jeff M. Phillips, and Vivek Srikumar. 2020. On measuring and mitigating biased inferences of word embeddings. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7659– 7666. find
  19. Lindsey DeWitt Prat, Olivia Nercy Ndlovu Lucas, Christopher Golias, and Mia Lewis. 2024. Decolonizing llms: An ethnographic framework for ai in african contexts. EPIC Proceedings, 2024(1):45–84. find
  20. Sunday B Fakunmoju and Funmi O Bammeke. 2017. Gender-based violence beliefs and stereotypes: Cross-cultural comparison across three countries. International Journal of Asian Social Science, 7:738– 753. find
  21. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, and 17 others. 2022. Red teaming…
  22. Axel Harneit-Seivers. 2007. Igbo history and society: The essays of adiele afigbo, edited by toyin falo-lamyth, history and society: The collected works of adiele afigbo, edited by toyin falola. African Affairs, 106:529–531. find
  23. Sukairaj Hafiz Imam, Babangida Sani, Dawit Ketema Gete, Bedru Yimam Ahamed, Ibrahim Said Ahmad, Idris Abdulmumin, Seid Muhie Yimam, Muhammad Yahuza Bello, and Shamsuddeen Hassan Muhammad. 2025. Automatic speech recognition for african low-resource languages: Challenges and future directions. Preprin… find
  24. Guido Ivetta, Marcos J. Gomez, Sofía Martinelli, Pietro Palombini, M. Emilia Echeveste, Nair Carolina Mazzeo, Beatriz Busaniche, and Luciana Benotti. 2025. Heseia: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in latin america. P… find
  25. Yacine Jernite, Huu Nguyen, Stella Biderman, Anna Rogers, Maraim Masoud, Valentin Danchev, Samson Tan, Alexandra Sasha Luccioni, Nishant Subramani, Isaac Johnson, Gerard Dupont, Jesse Dodge, Kyle Lo, Zeerak Talat, Dragomir Radev, Aaron Gokaslan, Somaieh Nikpoor, Peter Henderson, Rishi Bommasani, and… find
  26. Kyle Jerro. 2018. Linguistic complexity: A case study from swahili. find
  27. Akshita Jha, Aida Davani, Chandan K. Reddy, Shachi Dave, Vinodkumar Prabhakaran, and Sunipa Dev. 2023. Seegull: A stereotype benchmark with broad geo-cultural coverage leveraging generative models. Preprint, arXiv:2305.11840. find
  28. Hannah Kirk, Abeba Birhane, Bertie Vidgen, and Leon Derczynski. 2022. Handling and presenting harmful text in NLP research. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 497–510, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  29. Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro… — Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
  30. Victoria Wendy Lawson, Charity S. Akotia, and Maxwell Asumeng. 2015. Exploring Ethnic Stereotypes and Prejudice Among Some Major Ethnic Groups in Ghana. Journal of Social Science Studies, 2(1):17–35. find
  31. Juniel Shoko Matavire. 2024. Development of indigenous language orthographies: Setting up english as the torch bearer. International Journal of Critical Diversity Studies, 6(2). find
  32. Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Mas… find
  33. Elliot Mthembeni Mncwango and Nkosinathi Emmanuel Sithole. 2017. A diminished role of indigenous african languages in south african institutions. International Journal of Humanities and Social Science, 7(1):85–92. find
  34. Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Paul Röttger, Abigail Oppong, Andiswa Bukula, Chiamaka Ijeoma Chukwuneke, Ebrahim Chekol Jibril, Elyas Abdi Ismail, Esubalew Alemneh, Hagos Tesfahun Gebremichael, Lukma… find
  35. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…
  36. Joseph Gitile Naituli and Sellah Nasimiyu King’oro. 2018. An examination of ethnic stereotypes and coded language used in Kenya and its implication for national cohesion. International Journal of Development and Sustainability, 7(11):2670–2693. find
  37. Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa Berhe, Mofetoluwa A… — Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
  38. Jessica Ojo, Odunayo Ogundepo, Akintunde Oladipo, Kelechi Ogueji, Jimmy Lin, Pontus Stenetorp, and David Ifeoluwa Adelani. 2025. AfroBench: How good are large language models on African languages? In Findings of the Association for Computational Linguistics: ACL 2025, pages 19048–19095, Vienna, Aust… find
  39. Alexander Oluka. 2024. Mitigating biases in training data: Technical and legal challenges for sub-saharan africa. International Journal of Applied Research in Business and Management, 5(1):209–224. find
  40. Andrew Katumba, Chodrine Mutebi, Jagen Marvin, Eric Peter Wairagala, Mugizi Bruce, Peter Oketta, Lawrence Nderu, Obichi Obiajunwa, and 3 others. 2025. Amplify initiative: Building a localized data platform for globalized ai. Preprint, arXiv:2504.14105. — [arXiv:2504.14105](https://arxiv.org/abs/2504.14105) find
  41. Andrew Smart, Ben Hutchinson, Lameck Mbangula Amugongo, Suzanne Dikker, Alex Zito, Amber Ebinama, Zara Wudiri, Ding Wang, Erin van Liemt, João Sedoc, Seyi Olojo, Stanley Uwakwe, Edem Wornyo, Sonja Schmer-Galunder, and Jamila Smith-Loud. 2024. Socially responsible data for large multilingual language… — [arXiv:2409.05247](https://arxiv.org/abs/2409.05247) find
  42. Mesmin Tchindjang, Athanase Bopda, and Louise Angéline Ngamgne. 2008. Languages and cultural identities in africa. Museum International, 60(3):37–50. find
  43. Amanuel Isak Tewolde. 2024. Experiencing negative racial stereotyping: The case of coloured people in johannesburg, south africa. Social Sciences, 13:277. find
  44. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  45. Nedjma Ousidhoum, Meriem Beloucif, and Saif M. Mohammad. 2025. Building better: Avoiding pitfalls in developing language resources when data is scarce. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8881–8894, Vienna, Austria… find
  46. Akinpelu Ayokunnu Oyekunle. 2025. Exploring "linguistic complementarity" for intercultural communication in postcolonial african states. Frontiers in Communication, 10. find
  47. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105, Dublin, Ireland. Asso…
  48. Youssef Travaly and Kevin Muvunyi. 2020. The future is intelligent: Harnessing the potential of artificial intelligence in africa. The Brookings Institution. find
  49. Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651–1661, Florence, Italy.…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/verma-2026-safari-community-engaged-approach-dataset/