Elamur Research⌕
☰
home / papers / 2026 / LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross-Cultural Everyday Knowledge in LLMs

LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross-Cultural Everyday Knowledge in LLMs

Paper · 2026 · Ningjingke Ning

Summary

LocuPrompt is a system for SemEval-2026 Task 7, which uses an extended BLEnD benchmark to test whether LLMs know everyday, region-specific cultural facts across more than 30 language-region pairs. The system handles both tracks.

  • Short Answer Questions (SAQ): Questions are translated into English with Google Translate. The model is prompted to answer as a "local resident" of the target country, and the answer is translated back into the original language. Each locale is routed to whichever of five candidate LLMs scored best on the pilot data. Locales without pilot data borrow the choice of a linguistically or geographically close locale.
  • Multiple-Choice Questions (MCQ): The base model Qwen3-4B is adapted with QLoRA on the pilot data, using a similar locale-aware instruction.

On the pilot data, the English-pivot approach outperformed answering directly in the target language, and per-locale routing beat any single fixed model. However, fine-tuning on so few examples mostly taught the model the answer format rather than new cultural knowledge. The work offers a cheap, reproducible baseline that runs on a single GPU, and it shows that cultural gaps persist, especially for low-resource languages.

Key points

  • Data: about 148 pilot examples across 24 locales (roughly 6 per locale) served as the only development data. The hidden test set covers 31 locales, and no external training data was used.
  • SAQ pipeline: English-pivot generation reached 54.73% accuracy versus 45.95% for direct target-language generation. Per-locale routing among Llama3, Qwen, DeepSeek, GPT-5 and Gemini raised pilot accuracy from 52.0% to 54.7%. For locales absent from the pilot set, mappings such as en-US → en-GB and zh-TW → zh-CN were used.
  • MCQ setup: Qwen3-4B was fine-tuned with QLoRA (r = 16, α = 16, learning rate 2×10⁻⁴, 3 epochs, 8-bit AdamW) on a 24GB GPU, taking about 10 minutes per locale. Fine-tuning gained only 1.2 points over zero-shot, and 5-fold cross-validation averaged 27.13% (±7.16).
  • Language gap: English locales outscored non-English ones by a wide margin on SAQ (59.4% vs. 44.9%). On MCQ the gap was small (78.2% vs. 75.1%). Translation noise particularly hurt ga-IE, eu-ES and ar-MA.
  • Error analysis (200 sampled cases): missing regional facts caused 87% of SAQ errors and 82% of MCQ errors. Distractor confusion accounted for 17% of MCQ errors and translation artifacts for 10% of SAQ errors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Yunnan UniversityKunming · China17 works · 40 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Ningjingke Ning. 2026. LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross-Cultural Everyday Knowledge in LLMs. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.134

@inproceedings{ning2026locuprompt,
  title = {{LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross-Cultural Everyday Knowledge in LLMs}},
  author = {Ningjingke Ning},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.134},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.134}
}

References

17 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and *Table 5: Mapping from country code to the selected best-performing model for test submission. For locales absent from t…
  2. Country Code Country / Region Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. Better zero-shot reasoning with Selected Model role-play prompting. In arXiv preprint. arXiv. find
  3. EG MA SA BG GR AU GB EC ES MX IR FR IE ID JP KR SG LK PH CN ET DZ NG AS AZ US PV JB SE KP TW Egypt Morocco Saudi Arabia Bulgaria Greece Australia United Kingdom Ecuador Spain Mexico Iran France Ireland Indonesia Japan South Korea Singapore Sri Lanka Philippines China Ethiopia Algeria Nigeria India (… — IndoCulture: Exploring Geographically Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provinces
  4. Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. MKQA: A linguistically diverse benchmark for multilingual open domain question answering. In arXiv preprint. arXiv.
  5. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, and 1 others. 2024. Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages. Advances in Neural Information Processing Syste…
  6. Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros, Jiho Jin, Amr Keleg, Meriem Beloucif, Yi Zhou, Rodrigo Agerri, Vladimir Araujo, Naomi Baes, James Barry, Joanne Boisson, Nancy F. Chen, Christine de Kock, Aleksandra Edwards, Joseba Fernandez de Landa, Mohamed Fazli Imam, Huda Hakami, Shu-Kai Hsi… find
  7. Lindell Bromham and Keaghan J. Yaxley. 2023. Neighbours and relatives: accounting for spatial distribution when testing causal hypotheses in cultural evolution. Evolutionary Human Sciences, 5:e27. find
  8. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Preprint, arXiv:2305.14314.
  9. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Preprint, arXiv:2101.03961. find
  10. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. In arXiv preprint. arXiv.
  11. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. Preprint, arXiv:2306.02561.
  12. Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. 2024. Survey of cultural awareness in language models: Text and beyond. In arXiv preprint. arXiv.
  13. Edoardo M. Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020. XCOPA: A multilingual dataset for causal commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguisti…
  14. Sebastian Schuster, Sonal Gupta, Rushin Shah, and Mike Lewis. 2019. Cross-lingual transfer learning for multilingual task oriented dialog. In arXiv preprint. arXiv.
  15. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. Preprint, arXiv:1701.06538. find
  16. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  17. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ning-2026-locuprompt-semeval-2026-task-7/