Elamur Research⌕
☰
home / papers / 2026 / SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

Paper · 2026 · Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros et al.

Summary

LLMs often know little about everyday life outside Western, high-resource settings. Existing cultural benchmarks also tend to rely on monolingual or Wikipedia-style sources. This shared task tests how well systems handle culture-specific everyday knowledge using an expanded version of the BLEnD benchmark. The benchmark now covers 33 language–culture pairs, and 17 of them were added for this task. Many are low-resource locales spread over several continents. Native speakers wrote and answered the questions on topics such as food, sport, family, education, work and holidays, without help from LLMs or search engines. Language leads then aggregated and verified the answers.

There are two tracks: free-form short answers (SAQ) and multiple choice (MCQ). The data could be used only for evaluation, never for training or few-shot prompting. The task drew 62 final submissions and 19 system papers. MCQ proved much easier than SAQ, and results differed widely across locales. This makes the resource a useful testbed for measuring cultural misalignment, especially for under-represented languages.

Key points

  • Coverage: 17 new locales were added to the original BLEnD set, including Arabic varieties (Egypt, Morocco, Saudi Arabia), Irish, Basque, Tagalog, Japanese, Swedish, and Singaporean Mandarin, Malay and Tamil. The original BLEnD had about 15,000 short-answer questions.
  • Annotation: each new locale had at least five annotators who had lived at least half their lives in the region. They passed a 30-question pre-test and faced attention checks. Prolific was used only for Japanese, French and Australian English; for low-resource languages, the language leads recruited annotators directly.
  • Participation: over 140 people registered, 62 teams made final submissions, and 19 wrote system description papers. Track B (MCQ) attracted the most entries.
  • Main findings: basic prompting was enough for Singaporean English SAQ but not for Irish. RAG helped under-represented cultures such as Japan and Ethiopia but sometimes hurt results for the UK and US. Gains from activation steering depended strongly on the chosen layer and on prompt design.
  • Limitations: the authors note that exact-match scoring of short answers handles valid variants poorly in inflection-rich languages such as Arabic and Persian. They also note that culture is defined narrowly, along national borders.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBMArmonk · United States154 works · 72,606 citations🎓The University of MelbourneMelbourne · Australia102 works · 22,618 citations🎓University of the Basque CountryLeioa · Spain57 works · 16,520 citations🎓Uppsala UniversityUppsala · Sweden65 works · 13,164 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates138 works · 6,590 citations🎓Korea Advanced Institute of Science and TechnologyDaejeon · South Korea56 works · 3,269 citations🏛️Agency for Science, Technology and ResearchSingapore · Singapore31 works · 14,235 citations🎓Cardiff UniversityCardiff · United Kingdom23 works · 2,549 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros, Jiho Jin, Amr Keleg, Meriem Beloucif, Yi Zhou, Rodrigo Agerri, Vladimir Araujo, Naomi Baes, James Barry, Joanne Boisson, Nancy F. Chen, Christine de Kock, Aleksandra Edwards, Joseba Fernandez de Landa, Mohamed Fazli Imam, Huda Hakami, Shu-Kai Hsieh, Joseph Marvin Imperial, Roy Ka-Wei Lee, Zhengyuan Liu, Chenyang Lyu, Younes Samih, Johan Sjons, Bryan Tan, Asahi Ushio, Weihua Zheng, Alice Oh and Jose Camacho-Collados. 2026. SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.455

@inproceedings{ousidhoum2026semeval,
  title = {{SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures}},
  author = {Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros, Jiho Jin, Amr Keleg, Meriem Beloucif, Yi Zhou, Rodrigo Agerri, Vladimir Araujo, Naomi Baes, James Barry, Joanne Boisson, Nancy F. Chen, Christine {de Kock}, Aleksandra Edwards, Joseba Fernandez {de Landa}, Mohamed Fazli Imam, Huda Hakami, Shu-Kai Hsieh, Joseph Marvin Imperial, Roy Ka-Wei Lee, Zhengyuan Liu, Chenyang Lyu, Younes Samih, Johan Sjons, Bryan Tan, Asahi Ushio, Weihua Zheng, Alice Oh and Jose Camacho-Collados},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.455},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.455}
}

References

34 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Faisal Muhammad Adam, Lukman Jibril Aliyu, Sani Aji, Abdulhamid Abubakar, and Aliyu Rabiu Shuaibu. 2026. HausaNLP at SemEval-2026 Task 7: Prompt-based Hausa Cultural Question Answering. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  2. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling “culture” in LLMs: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lan…
  3. Isaac Nyadu Adjei and Saurav K. Aryal. 2026. Howard University-AI4PC at SemEval-2026 Task 7: Culturally Aware Multilingual Model Routing Through a Mixture-of-Specialists Framework. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  4. Yusser Al Ghussin, Daniil Gurgurov, Yasser Hamidullah, Josef van Genabith, Cristina España-Bonet, and Simon Ostermann. 2026. DFKI-MLT at SemEval-2026 Task 7: Steering Multilingual Models Towards Cultural Knowledge. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-202… find
  5. Mai Alkhamissi, Yunze Xiao, Badr AlKhamissi, and Mona T. Diab. 2026. Hire your anthropologist! rethinking culture benchmarks through an anthropological lens. In Findings of the Association for Computational Linguistics: EACL 2026, pages 1218–1235. Association for Computational Linguistics. find
  6. Danileth Almanza, Jairo Serrano, Edwin Puertas, and Juan Carlos Martinez Santos. 2026. VerbaNexAI at SemEval-2026 Task 7: Integrating Web Snippets and RAG for the Evaluation of Multilingual Cultural Knowledge in LLMs. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-… find
  7. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — Qwen Technical Report
  8. Liliia Bogdanova, Shiran Sun, Lifeng Han, Natalia Amat-Lefort, and Flor Miriam Plaza-del Arco. 2026. find
  9. FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  10. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388.
  11. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. Massively multi-cultural knowledge acquisition & lm benchmarking. arXiv preprint arXiv:2402.09369.
  12. Jianning Gao, Xianling Mao, Shumin Shi, Duanzhi Zhaxi, Yingbo Sun, Xiandeng Li, and Binyang Li. 2026. uir-cis-7 at SemEval-2026 Task 7: Zero-Shot Chain-of-Thought Reasoning for Cross-Cultural Daily Knowledge. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  13. Reihaneh Iranmanesh, Ophir Frieder, and Nazli Goharian. 2026. GUIR at SemEval-2026 Task 7: Probing Cultural Knowledge in LLMs via Multi-Agent Debate. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  14. Meizhi Jin, Zhichao Meng, Junqi Yin, Lianxin Jiang, and Jianyu Li. 2026. king001 at SemEval-2026 Task 7: Cross-Language Cultural Everyday Knowledge Q&A System Based on RAG. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  15. Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki, and Timothy Baldwin. 2024. Evaluating gender bias in large language models via chain-of-thought prompting. arXiv preprint arXiv:2401.15585. — [arXiv:2401.15585](https://arxiv.org/abs/2401.15585) find
  16. Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024. Click: A benchmark dataset of cultural and linguistic intelligence in korean. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024… find
  17. Fajri Koto, Rahmad Mahendra, Nurul Aisyah, and Timothy Baldwin. 2024. Indoculture: Exploring geographically-influenced cultural commonsense reasoning across eleven indonesian provinces. arXiv preprint arXiv:2404.01854.
  18. Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2025. Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art. Preprint, arXiv:2406.03930.
  19. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie… — BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
  20. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2024. Having beer after prayer? measuring cultural bias in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16366–16393, Bangkok, Thailand. Associatio…
  21. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Information Quality, 15(2):1–21. find
  22. Jingke Ning. 2026. LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross-Cultural Everyday Knowledge in LLMs. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  23. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — GPT-4 Technical Report
  24. Nedjma Ousidhoum, Meriem Beloucif, and Saif M. Mohammad. 2025. Building better: Avoiding pitfalls in developing language resources when data is scarce. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8881–8894, Vienna, Austria… find
  25. Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. 2024. Survey of cultural awareness in language models: Text and beyond. Preprint, arXiv:2411.00860.
  26. Mohammad Marufur Rahman, Rakshitha Rao Ailneni, and Sanda Harabagiu. 2026. UTD-HLTRI at SemEval-2026 Task 7: Bridging Cultural Knowledge Gaps in LLMs via Web-Augmented Context. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  27. Aditya Singh and Rickarya Das. 2026. CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  28. Guijin Son, Hanwool Lee, Suwan Kim, Huiseo Kim, Jae cheol Lee, Je Won Yeom, Jihyu Jung, Jung woo Kim, and Songseong Kim. 2024. Hae-rae bench: Evaluation of korean knowledge in language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources… find
  29. Jiwoo Song, Sihyeong Yeom, and Harksoo Kim. 2026. K-NLPers at SemEval-2026 Task 7: Multiple LLM Agent Debate System for Everyday Knowledge Across Diverse Languages and Cultures. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  30. Swetha Krishna Sriram and Nirupama Sekar. 2026. Models Without Borders at SemEval-2026 Task 7: Bridging Cultural Contexts with Search-Grounded QA. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  31. Cheng Tang, Zhichao Meng, and Meizhi Jin. 2026. chengtang at SemEval-2026 Task 7: A Retrieval-Augmented Generation Framework for Cultural Perspective Alignment in Everyday MCQs. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  32. Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, and Jafar Razmara. 2026. Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  33. Kongqiang Wang, Peng Zhang, and Qingli Tan. 2026. Wangkongqiang at SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  34. Yen Yee Yam and Hong Meng Yam. 2026. Pinetree at SemEval-2026 Task 7: A Large-Scale Failure Analysis of Cultural Grounding in Language Models. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ousidhoum-2026-semeval-2026-task-7-everyday/