Elamur Research⌕
☰
home / papers / 2024 / Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval

Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval

Summary

Adapting a recipe to another culture means keeping the identity of the dish while matching the ingredients, tastes and writing conventions of the new audience. Generative models such as GPT-4 often fall short here. For example, they may keep ingredients that are uncommon in the target cuisine. The authors instead treat adaptation as a search problem: given a Chinese recipe, find fitting recipes written within English-speaking culinary traditions. They define this new task with a three-level relevance scale and detailed judging guidelines. They also build a manually annotated Chinese-English benchmark, pooling candidates from several retrieval models under a limited annotation budget. Their framework, CARROT, requires no fine-tuning. LLMs rewrite the query, multiple retrievers produce ranked lists, and an LLM re-ranks the merged results by relevance and cultural fit. CARROT outperforms standard retrievers. Compared with LLM-generated adaptations, its retrieved recipes are more consistent and culturally appropriate. The work provides both an evaluation resource and a retrieval-based route to cultural adaptation in NLP.

Key points

  • Task definition: a source-culture recipe serves as the query and target-culture recipes serve as documents. Relevance is graded as 0 (Not Match), 1 (Partial Match) or 2 (Exact Match). Partial matches must belong to the same dish category.
  • Challenges addressed:
    • Title mismatch caused by naming conventions (e.g., Kung Pao Chicken), ingredient substitution (taro → potato) and culture-specific food knowledge (地三鲜).
    • Scarcity of labeled data.
    • The need to rank by cultural appropriateness as well as relevance.
  • CARROT pipeline: machine-translated titles are combined with two LLM rewrites. One rewrite regenerates the title from ingredients and steps, inspired by doc2query. The other culturally adapts the title. LLM re-ranking then follows. Tested LLMs include Llama3, Qwen and Baichuan, with Llama3 performing best.
  • Results: CARROT-Llama3 scored best among retrieval methods on ROUGE-L, BertScore, P@1, Preservation and Consistency, and came close to the best on Cultural Appropriateness. The baseline was a translated title with a bi-encoder MPNet. Retrieval also yields diverse results; for red bean soup, the top 10 contained more than five distinct main-ingredient combinations.
  • Availability and limitations: code and data are available at github.com/TenneyHu/CARROT. Limitations include a small benchmark, coverage of only Chinese-English, and no domain fine-tuning of the retrievers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.emnlp-main.61

@inproceedings{hu2024bridging,
  title = {{Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval}},
  author = {Hu, Tianyi and Maistro, Maria and Hershcovich, Daniel},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.61},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.61}
}

References

34 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… find our card
  2. Baichuan. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305. find our card
  3. Michał Bień, Michał Gilski, Martyna Maciejewska, Wojciech Taisner, Dawid Wisniewski, and Agnieszka Lawrynowicz. 2020. RecipeNLG: A cooking recipes dataset for semi-structured text generation. In Proceedings of the 13th International Conference on Natural Language Generation, pages 22–28, Dublin, Ir… find
  4. Andrea Borghini. 2015. What is a recipe? Journal of Agricultural and Environmental Ethics, 28:719–738. find
  5. Yong Cao, Yova Kementchedjhieva, Ruixiang Cui, Antonia Karamolegkou, Li Zhou, Megan Dare, Lucia Donatelli, and Daniel Hershcovich. 2024. Cultural Adaptation of Recipes. Transactions of the Association for Computational Linguistics, 12:80–99. find
  6. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Pi-queras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022. Challenges and strategies in cross-cultural… find our card
  7. Jaana Kekäläinen and Kalervo Järvelin. 2002. Using graded relevance assessments in ir evaluation. Journal of the American Society for Information Science and Technology, 53(13):1120–1129. find
  8. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 39– 48. find our card
  9. Egon P Köster. 2009. Diversity in the determinants of food choice: A psychological perspective. Food quality and preference, 20(2):70–82. find
  10. Yen-Chieh Lien, Hamed Zamani, and W Bruce Croft. 2020. Recipe retrieval with visual query of ingredients. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1565–1568. find
  11. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics. find
  12. Xiao Liu, Yansong Feng, Jizhi Tang, Chengang Hu, and Dongyan Zhao. 2022. Counterfactual recipe generation: Exploring compositional generalization in a realistic scenario. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7354–7370, Abu Dhabi, United Ara… find
  13. Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. arXiv preprint arXiv:1904.08375. find our card
  14. Shramay Palta and Rachel Rudinger. 2023. Fork: A bite-sized test set for probing culinary cultural biases in commonsense reasoning models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9952–9962. find
  15. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… find our card
  16. V Pavlu and J Aslam. 2007. A practical sampling strategy for efficient retrieval evaluation. College of Computer and Information Science, Northeastern University. find
  17. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find our card
  18. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. find
  19. Nils Reimers and Iryna Gurevych. 2020a. Making monolingual sentence embeddings multilingual using knowledge distillation. arXiv preprint arXiv:2004.09813. find our card
  20. Nils Reimers and Iryna Gurevych. 2020b. Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4512–4525, Online. Association for Computational Linguistics. find our card
  21. Paul Rozin. 1996. The socio-cultural context of eating and food choice. In Food choice, acceptance and consumption, pages 83–104. Springer. find
  22. Tetsuya Sakai and Noriko Kando. 2008. On information retrieval metrics designed for evaluation with incomplete relevance assessments. Information Retrieval, 11:447–470. find
  23. Amaia Salvador, Erhan Gundogdu, Loris Bazzani, and Michael Donoser. 2021. Revamping cross-modal recipe retrieval with hierarchical transformers and self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15475–15484. find
  24. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2021. Colbertv2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint arXiv:2112.01488. find our card
  25. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems, 33:16857– 16867. find our card
  26. Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023. Is chatgpt good at search? investigating large language models as re-ranking agent. arXiv preprint arXiv:2304.09542. find our card
  27. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  28. Zhonghe Zhang, Xiaoyu He, Vivek Iyer, and Alexandra Birch. 2024. Cultural adaptation of menus: A fine-grained approach. arXiv preprint arXiv:2408.13534. find
  29. Kentaro Takiguchi, Mikhail Fain, Niall Twomey, and Luis M Vaquero. 2021. Evaluation of field-aware neural ranking models for recipe search. arXiv preprint arXiv:2105.05710. — [arXiv:2105.05710](https://arxiv.org/abs/2105.05710) find
  30. Li Zhou, Taelin Karidi, Nicolas Garneau, Yong Cao, Wanlong Liu, Wenyu Chen, and Daniel Hershcovich. 2024. Does mapo tofu contain coffee? probing llms for food-related cultural knowledge. arXiv preprint arXiv:2404.06833. find
  31. Yanran Tang, Ruihong Qiu, and Xue Li. 2023. Prompt-based effective input reformulation for legal case retrieval. In Australasian Database Conference, pages 87–100. Springer. find
  32. Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen. 2023. Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107. find our card
  33. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480, Lisboa, Portugal. European Association for Machine Translation. find
  34. Damir Dennis Torrico, Sigfredo Fuentes, Claudia Gonzalez Viejo, Hollis Ashman, and Frank R Dunshea. 2019. Cross-cultural effects of food product familiarity on sensory acceptability and non-invasive physiological responses of consumers. Food research international, 115:439–450. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/hu-2024-bridging-cultures-kitchen-framework-benchmark/