Elamur Research⌕
☰
home / papers / 2025 / WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

Paper · 2025 · Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan et al.

Summary

Vision Language Models (VLMs) often lack culture-specific knowledge, especially outside English and for underrepresented regions. WorldCuisines tests this using food as a cultural proxy. The authors built WC-KB, a knowledge base of culturally distinctive dishes drawn from Wikipedia, with openly licensed images from Wikimedia Commons and annotated metadata. From it they generated WC-VQA, a parallel visual question answering dataset whose question templates were translated by native speakers, including dialects and formal and casual registers.

The benchmark has two tasks: naming a dish from its image, and identifying where it originates or is commonly eaten. The dish-naming task has three variants: no context, correct context, and misleading (adversarial) context. Answers are either multiple-choice or open-ended.

Models do better when given the right location context but are easily misled by adversarial hints. They also struggle with specific regional cuisines and low-resource languages. The released data, code and leaderboard support culturally aware multilingual VLM evaluation.

Key points

  • WC-VQA has over 1 million text-image pairs (about 1.15M in total) across 30 languages and dialects from 9 language families. The authors describe it as the largest multicultural VQA benchmark so far.
  • There are two evaluation splits (12k and 60k instances) and a training split of roughly 1M instances. Everything is released under CC BY-SA 4.0 on Hugging Face and GitHub.
  • WC-KB covers 2,414 dishes with 6,045 images from 189 countries, concentrated mostly in Asia, Europe and North America. It also includes 90 multilingual prompt templates and 401 parallel location and cuisine entries.
  • Among open-source models, Llama 3.2 Instruct performed best overall. NVLM-D trailed its base model, Qwen2 VL Instruct, possibly because NVLM is tuned heavily toward English. Larger models generally scored higher.
  • Known limitations:
    • Dishes come only from English Wikipedia.
    • Evaluation relies on accuracy and BERTScore, and the authors say metrics suited to multicultural VQA are still an open problem.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓Monash UniversityMelbourne · Australia60 works · 54,866 citations🎓Hong Kong Polytechnic UniversityHong Kong · China84 works · 4,956 citations🎓Korea Advanced Institute of Science and TechnologyDaejeon · South Korea51 works · 3,167 citations🔬Canadian Institute for Advanced ResearchToronto · Canada28 works · 16,725 citations🎓Tokyo Institute of TechnologyTokyo · Japan22 works · 2,137 citations🎓Cardiff UniversityCardiff · United Kingdom22 works · 2,444 citations🎓Boston UniversityBoston · United States26 works · 4,022 citations🎓Japan Advanced Institute of Science and TechnologyNomi Shi · Japan7 works · 980 citations🎓Ontario Tech UniversityOshawa · Canada8 works · 1,554 citations🏛️AI SingaporeSingapore · Singapore5 works · 383 citations🎓University of LagosLagos · Nigeria2 works · 477 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 4Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksMultilingual & cross-lingual NLP · 828 worksVision & language · 657 worksQuestion answering · 441 works

Cite

Cite as Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Wang Yutong, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Cheng Ching Lam, Daud Abolade, Emmanuele Chersoni, Enrico Santus, Fariz Ikhwantri, Garry Kuwanto, Hanyang Zhao, Haryo Akbarianto Wibowo, Holy Lovenia, Jan Christian Blaise Cruz, Jan Wira Gotama Putra, Junho Myung, Lucky Susanto, Maria Angelica Riera Machin, Marina Zhukova, Michael Anugraha, Muhammad Farid Adilazuarda, Natasha Christabelle Santosa, Peerat Limkonchotiwat, Raj Dabre, Rio Alexander Audino, Samuel Cahyawijaya, Shi-Xiong Zhang, Stephanie Yulia Salim, Yi Zhou, Yinxuan Gui, David Ifeoluwa Adelani, En-Shiun Annie Lee, Shogo Okada, Ayu Purwarianti, Alham Fikri Aji, Taro Watanabe, Derry Tanti Wijaya, Alice Oh and Chong-Wah Ngo. 2025. WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.naacl-long.167

@inproceedings{winata2025worldcuisines,
  title = {{WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines}},
  author = {Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Wang Yutong, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Cheng Ching Lam, Daud Abolade, Emmanuele Chersoni, Enrico Santus, Fariz Ikhwantri, Garry Kuwanto, Hanyang Zhao, Haryo Akbarianto Wibowo, Holy Lovenia, Jan Christian Blaise Cruz, Jan Wira Gotama Putra, Junho Myung, Lucky Susanto, Maria Angelica Riera Machin, Marina Zhukova, Michael Anugraha, Muhammad Farid Adilazuarda, Natasha Christabelle Santosa, Peerat Limkonchotiwat, Raj Dabre, Rio Alexander Audino, Samuel Cahyawijaya, Shi-Xiong Zhang, Stephanie Yulia Salim, Yi Zhou, Yinxuan Gui, David Ifeoluwa Adelani, En-Shiun Annie Lee, Shogo Okada, Ayu Purwarianti, Alham Fikri Aji, Taro Watanabe, Derry Tanti Wijaya, Alice Oh and Chong-Wah Ngo},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.167},
  url = {https://doi.org/10.18653/v1/2025.naacl-long.167}
}

References

7 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219.
  2. Marié Abe. 2013. Tokyo, japan. In The Ethnomusicolo-gists’ Cookbook, pages 40–45. Routledge. find
  3. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  4. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling" culture" in llms: A survey. arXiv preprint arXiv:2403.15412.
  5. Pulkit Agarwal, Settaluri Sravanthi, and Pushpak Bhattacharyya. 2024. Indifoodvqa: Advancing visual question answering and reasoning with a knowledge-infused synthetic data generation pipeline. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1158–1176. find
  6. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Devendra Chaplot, Jessica Chudnovsky, Saurabh Garg, Theophile Gervet, Soham Ghosh, Amélie Héliou, Paul Jacob, et al. 2024. Pixtral 12b. arXiv preprint arXiv:2410.07073. find
  7. Gina M Almerico. 2014. Food and identity: Food studies, cultural, and personal identity. Journal of International Business and Cultural Studies, 8:1. \#\# Ethical Considerations find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/winata-2025-worldcuisines-massive-scale-benchmark-multilingual/