Elamur Research⌕
☰
home / papers / 2026 / Afri-MCQA: Multimodal Cultural Question Answering for African Languages

Afri-MCQA: Multimodal Cultural Question Answering for African Languages

Paper · 2026 · Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva et al.

Summary

Afri-MCQA addresses a gap in multimodal AI evaluation. Existing cultural VQA benchmarks cover few African languages and query models only through text, even though many African languages are used mainly in spoken form. The authors built a benchmark of about 7.5k image-grounded question–answer pairs in 15 sub-Saharan languages from 12 countries, roughly 500 per language. Native speakers hired through Upwork created all items. Each item is parallel in English and the native language, in both text and recorded audio. Quality control had two stages: a pilot round to screen annotators, then a main round checked by native-speaker language coordinators.

The authors tested several open and proprietary MLLMs. Open-weight models scored very poorly, approaching zero on open-ended questions asked in a native language or through speech. English queries always did better than native ones. Control experiments suggest that weak language understanding, more than missing cultural knowledge, is the main bottleneck. The results argue for speech-first modeling, pretraining on cultural content, and cross-lingual transfer of cultural knowledge.

Key points

  • Data and annotation: 15 languages (e.g., Amharic, Hausa, Yoruba, Zulu, Tigrinya, Kinyarwanda) with about 392.6M combined speakers.
    • Images fall into the 10 categories defined by CVQA.
    • Per language, annotators first submitted 50 pilot items, then 450 more in the main phase.
  • Speech: the benchmark includes audio in each native language and in African-accented English. English audio was not recorded for Sesotho and Zulu.
  • Task format: Gemini-2.5-Pro scored 78% on multiple-choice VQA but only 38% on Open-VQA. Some smaller models scored below 10% on Open-VQA.
  • Language gaps:
    • The English-over-native advantage ranged from 2% (Gemini-2.5-Pro) to 19% (Qwen).
    • Control tests on AfriXNLI and AfriMMLU showed larger language-understanding gaps (13–47%).
    • Speech failures came from weak language identification (2–4% for Qwen) and ASR WER of 85–100%+ for non-Gemini models.
  • Scoring and release: open answers were judged by GPT-4o-mini for semantic matching only, cross-checked with chrF++ and human evaluation. The dataset is released on Hugging Face as Atnafu/Afri-MCQA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Vision & language · 811 worksQuestion answering · 487 works

Cite

Cite as Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Muhidin A. Mohamed, Debela Desalegn Yadeta, Negasi Haile Abadi, Abigail Oppong, Nnaemeka Casmir Obiefuna, Idris Abdulmumin, Naome A Etori, Eric Peter Wairagala, Kanda Patrick Tshinu, Imanigirimbabazi Emmanuel, Gabofetswe Malema, Alham Fikri Aji, David Ifeoluwa Adelani and Thamar Solorio. 2026. Afri-MCQA: Multimodal Cultural Question Answering for African Languages. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1869

@inproceedings{tonja2026afri,
  title = {{Afri-MCQA: Multimodal Cultural Question Answering for African Languages}},
  author = {Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Muhidin A. Mohamed, Debela Desalegn Yadeta, Negasi Haile Abadi, Abigail Oppong, Nnaemeka Casmir Obiefuna, Idris Abdulmumin, Naome A Etori, Eric Peter Wairagala, Kanda Patrick Tshinu, Imanigirimbabazi Emmanuel, Gabofetswe Malema, Alham Fikri Aji, David Ifeoluwa Adelani and Thamar Solorio},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1869},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1869}
}

References

35 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 oth… — Phi-4 Technical Report
  2. Ife Adebara and Muhammad Abdul-Mageed. 2022. Towards afrocentric NLP for African languages: Where we are and where we can go. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3814–3841, Dublin, Ireland. Association for Computat… find
  3. David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and En-Shiun Annie Lee. 2024. SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects. In Proceedings of the 18th Conference of the Eu…
  4. David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Alabi, Atnafu Lambebo Tonja, Christine Mwase, Odunayo Ogundepo, Bonaventure F. P. Dossou, Akintunde Oladipo, Doreen Nixdorf, Chris Chinenye Emezue, Sana Al-azzawi, Blessing Sibanda, Davis David, Lolwethu Ndolela, Jonathan Mukiibi, Tu… — MasakhaNEWS: News Topic Classification for African languages
  5. David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kab… — IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
  6. Jinzheng He Hangrui Hu Ting He Shuai Bai Keqin Chen Jialin Wang Yang Fan Kai Dang Bin Zhang Xiong Wang Yunfei Chu Junyang Lin Jin Xu, Zhifang Guo. 2025. Qwen2.5-omni technical report. arXiv preprint arXiv:2503.20215.
  7. Jesujoba Oluwadara Alabi, Michael A. Hedderich, David Ifeoluwa Adelani, and Dietrich Klakow. 2025. Charting the landscape of African NLP: Mapping progress and shaping the road ahead. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27807–27841, Suzhou,… find
  8. Klaus Krippendorff. 2011. Computing krippendorff’s alpha-reliability. find
  9. Israel Abebe Azime, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Yonas Chanie, Bontu Fufa Balcha, Negasi Haile Abadi, Henok Biadglign Ademtew, Mulubrhan Abebe Nerea, Debela Desalegn Yadeta, Derartu Dagne Geremew, Assefa Ats-biha Tesfu, Philipp Slusallek, Thamar Solorio, and Dietrich Klakow. 2025. Pro… find
  10. Israel Abebe Azime, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Mitiku Yohannes Fuge, Aman Kassahun Wassie, Eyasu Shiferaw Jada, Yonas Chanie, Walelign Tewabe Sewunetie, and Seid Muhie Yimam. 2024. Walia-LLM: Enhancing Amharic-LLaMA by integrating task-specific and generative datasets. In Findings o…
  11. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  12. Harald Hammarström. 2018. A survey of african languages. find
  13. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  14. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu,… find
  15. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021a. Visually grounded reasoning across languages and cultures. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10467–10485, Online and Punta Cana… find
  16. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021b. Visually grounded reasoning across languages and cultures. arXiv preprint arXiv:2109.13238. find
  17. Rada Mihalcea, Oana Ignat, Longju Bai, Angana Borah, Luis Chiruzzo, Zhijing Jin, Claude Kwizera, Joan Nwatu, Soujanya Poria, and Thamar Solorio. 2025. Why ai is weird and shouldnt be this way: Towards ai for everyone, with everyone, by everyone. find
  18. Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Paul Röttger, Abigail Oppong, Andiswa Bukula, Chiamaka Ijeoma Chukwuneke, Ebrahim Chekol Jibril, Elyas Abdi Ismail, Esubalew Alemneh, Hagos Tesfahun Gebremichael, Lukma… find
  19. Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd Van Steenkiste, Lisa Anne Hendricks, Aishwarya Agrawal, and 1 others. 2024. Benchmarking vision language models for cultural understanding. arXiv preprint arXiv:2407.10920.
  20. James Cross Onur Çelebi Maha Elbayad Kenneth Heafield Kevin Heffernan Elahe Kalbassi Janice Lam Daniel Licht Jean Maillard Anna Sun Skyler Wang Guillaume Wenzek Al Youngblood Bapi Akula Loic Barrault Gabriel Mejia Gonzalez Prangthip Hansanti John Hoffman Semarley Jarrett Kaushik Ram Sadagopan Dirk R… find
  21. Derek Nurse and Gerard Philippson, editors. 2006. The Bantu Languages. Routledge Language Family Series. Routledge, London, England. find
  22. Odunayo Ogundepo, Tajuddeen R Gwadabe, Clara E Rivera, Jonathan H Clark, Sebastian Ruder, David Ifeoluwa Adelani, Bonaventure FP Dossou, Abdou Aziz Diop, Claytone Sikasote, Gilles Hacheme, and 1 others. 2023. Afriqa: Cross-lingual open-retrieval question answering for african languages. arXiv prepri… find
  23. Jessica Ojo, Odunayo Ogundepo, Akintunde Oladipo, Kelechi Ogueji, Jimmy Lin, Pontus Stenetorp, and David Ifeoluwa Adelani. 2025. Afrobench: how good are large language models on african languages? In Findings of the Association for Computational Linguistics: ACL 2025, pages 19048–19095. find
  24. Shantipriya Parida, Idris Abdulmumin, Shamsuddeen Hassan Muhammad, Aneesh Bose, Guneet Singh Kohli, Ibrahim Said Ahmad, Ketan Kotwal, Sayan Deb Sarkar, Ondřej Bojar, and Habeebah Adamu Kakudi. 2023. Havqa: A dataset for visual question answering and multimodal research in hausa language. arXiv prep… find
  25. Maja Popović. 2015. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the tenth workshop on statistical machine translation, pages 392–395.
  26. David Romero, Chenyang Lyu, Haryo Wibowo, Santiago Góngora, Aishik Mandal, Sukannya Purkayastha, Jesus-German Ortiz-Barajas, Emilio Cueva, Jinheon Baek, Soyeong Jeong, and 1 others. 2024. Cvqa: Culturally-diverse multilingual visual question answering benchmark. Advances in Neural Information Proces…
  27. Florian Schneider and Sunayana Sitaram. 2024. M5–a diverse benchmark to assess the performance of large multimodal models across multilingual and multicultural vision-language tasks. arXiv preprint arXiv:2407.03791. find
  28. Belay Simane, Thandi Kapwata, Natasha Naidoo, Guéladio Cissé, Caradee Y. Wright, and Kiros Berhane. 2025. Ensuring africas food security by 2050: The role of population growth, climate-resilient strategies, and putative pathways to resilience. Foods, 14(2):262. find
  29. Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David I Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, and 1 others. 2024. Global mmlu: Understanding and addressing cultural and linguistic biases in multilingual evaluation…
  30. Atnafu Lambebo Tonja, Israel Abebe Azime, Tadesse Destaw Belay, Mesay Gemeda Yigezu, Moges Ahmed Ah Mehamed, Abinew Ali Ayele, Ebrahim Chekol Jibril, Michael Melese Woldey-ohannis, Olga Kolesnikova, Philipp Slusallek, and 1 others. 2024a. Ethiollm: Multilingual large language models for ethiopian la… find
  31. Atnafu Lambebo Tonja, Bonaventure FP Dossou, Jessica Ojo, Jenalea Rajab, Fadel Thior, Eric Peter Wairagala, Anuoluwapo Aremu, Pelonomi Moiloa, Jade Abbott, Vukosi Marivate, and 1 others. 2024b. Inkubalm: A small language model for low-resource african languages. arXiv preprint arXiv:2408.17024. — [arXiv:2408.17024](https://arxiv.org/abs/2408.17024) find
  32. Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Amandeep Kumar, Kartik Kukreja, and 1 others. 2025a. All languages matter: Evaluating lmms on culturally diverse 100 languages. In Proceedings of the Computer… find
  33. Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Amandeep Kumar, Kartik Kukreja, and 1 others. 2025b. All languages matter: Evaluating lmms on culturally diverse 100 languages. In Proceedings of the Computer… find
  34. Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Wang Yutong, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Cheng Ching Lam, Daud Abola… — WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
  35. Hao Yu, Jesujoba Oluwadara Alabi, Andiswa Bukula, Jian Yun Zhuang, En-Shiun Annie Lee, Tadesse Kebede Guge, Israel Abebe Azime, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither M… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tonja-2026-afri-mcqa-multimodal-cultural-question/