Elamur Research⌕
☰
home / papers / 2026 / Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning

Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning

Paper · 2026 · Mena Attia, Mona Diab and Thamar Solorio

Summary

Linguists and cognitive scientists have long argued that understanding figurative language depends on cultural familiarity. This paper tests whether the same coupling appears inside LLMs. The authors run a controlled fine-tuning study in Arabic with four models: ALLaM-7B, Fanar-1-9B, Qwen3-8B and Llama-3.1-8B. They test transfer in both directions, from cultural data to figurative understanding and the reverse, and also across figurative forms. Training data comes from ArabCulture, Palm (with proverb items removed), FannOrFlop poetry and Jawaher proverbs. Evaluation uses AraDiCE-Culture, the Jawaher test split and Kinayat Egyptian idioms. An ArabicMMLU language and grammar condition serves as a control for plain Arabic exposure.

Only one effect holds up: poetry fine-tuning improves idiom comprehension, and the control condition does not show this gain. Cultural fine-tuning actually hurts proverb interpretation in the Arabic-centric models. Most other shifts are within noise. The result matters because it warns that two closely related kinds of knowledge do not automatically support each other through fine-tuning.

Key points

  • The only reliable transfer is poetry (FannOrFlop) → idioms (Kinayat): +2.33%, p = 0.021. The ArabicMMLU control (980 examples) regresses on the same benchmark.
  • Each condition trains with LoRA on a 1,000-sample subset (Jawaher has only 800 training samples). Every run is repeated with three seeds that shuffle the multiple-choice options.
  • The Arabic-centric models (ALLaM, Fanar) start from higher baselines and lose accuracy on proverbs after cultural fine-tuning. The worst single run drops by 5.56% (Palm, seed 21).
  • The multilingual models (Llama, Qwen) have more room to improve. Poetry tuning gives Llama its best single-seed idiom result: +10.00%, with a net +31 correct predictions across seeds.
  • Error analysis shows that fine-tuning strengthens experiential cultural knowledge (food, customs, celebrations) but weakens historical and domain-specific factual knowledge.

Interesting ideas

  • I1. The authors add a control condition trained on generic in-language data, which separates "the content helped" from "more exposure to the language helped."
  • I2. They remove items from training sets that overlap the target phenomenon (proverbs in Palm), so no condition secretly carries the supervision it is meant to test.
  • I3. Results are reported as net counts of fixed minus broken predictions, alongside accuracy, which exposes hidden regressions behind a small positive average.
  • I4. Re-running evaluation with shuffled multiple-choice option order separates real gains from position effects.
  • I5. Figurative skill seemed to transfer between non-literal forms (poetry to idioms) but not from cultural facts, suggesting that sensitivity to non-literal meaning is a separate capacity.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): any new training set for gparse or the tagger could be paired with a size-matched generic-text control, so that improvements are credited to the content rather than to volume.
  • I2 (learn): when silver data or teacher-generated examples are used to test transfer to irony or idioms, examples containing those phenomena should be filtered out first to keep the experiment clean.
  • I3 (answer): evaluations on school tests and fable QA could log per-item flips between model versions, revealing which readers gain on some questions while silently breaking others.
  • I4 (answer): multiple-choice school-course tests could be scored over several option permutations, so that answer-position bias is caught as a negative control.
  • I5 (frames): the irony and implicature modules in prag could share one literal-versus-intended mismatch detector across idioms, metaphor and proverbs, instead of building a separate lexicon for each.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States365 works · 158,048 citations

Also on the byline: Routledge.

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

🧭 Topics 2 Language models & pretraining · 1,343 works Semitic languages · 1,484 works

Cite

Cite as Mena Attia, Mona Diab and Thamar Solorio. 2026. Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning. arXiv:2608.18361.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2608.18361 · arXiv 2608.18361

@misc{attia2026figurative,
  title = {{Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning}},
  author = {Mena Attia, Mona Diab and Thamar Solorio},
  year = {2026},
  doi = {10.48550/arxiv.2608.18361},
  eprint = {2608.18361},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2608.18361}
}

References

38 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Farid Adilazuarda, Chen Cecilia Liu, Iryna Gurevych, and Alham Fikri Aji. 2025. From surveys to narratives: Rethinking cultural value adaptation in LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 18052–18079, Suzhou, China. Association for Computational Linguistics. find
  2. Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, Yousef Ali, and Maged Al-shaibani. 2024. CIDAR: Culturally relevant instruction dataset for Arabic. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12878–12901, Bangkok, Thailand. Association for Computational Linguistics. ↗
  3. Wafa Alghallabi, Ritesh Thawkar, Sara Ghaboura, Ketan More, Omkar Thawakar, Hisham Cholakkal, Salman Khan, and Rao Muhammad Anwer. 2025. Fann or flop: A multigenre, multiera benchmark for arabic poetry understanding in llms. Preprint, arXiv:2505.18152. ↗
  4. Doğukan Arslan, Rodrigo Wilkens, Wei He, Dilara Torunoglu Selamet, Thomas Pickard, Aline Villavicencio, Adriana Silvina Pagano, and Gülşen Eryiğit. 2026. MWE-2026 shared task: AdMIRe 2 advancing multimodal idiomaticity representation. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), pages 276–287, Rabat, Marocco. Association for Computational Linguistics. Linguistics (Volume 1: Long Papers), pages 25663– 25701, Vienna, Austria. Association for Computational Linguistics. find
  5. Mena Attia, Aashiq Muhamed, Mai Alkhamissi, Thamar Solorio, and Mona T. Diab. 2026. Beyond understanding: Evaluating the pragmatic gap in LLMs’ cultural processing of figurative language. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7238–7265, Rabat, Morocco. Association for Computational Linguistics. ↗
  6. Dmitrij Olegovich Dobrovol’skij and Elisabeth Piirainen. 2022. Figurative language : Cross-cultural and cross-linguistic perspectives, 2 edition. de Gruyter Mouton. find
  7. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others. 2024. The language model evaluation harness. find
  8. Israel Abebe Azime, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Yonas Chanie, Bontu Fufa Balcha, Negasi Haile Abadi, Henok Biadglign Ademtew, Mulubrhan Abebe Nerea, Debela Desalegn Yadeta, Derartu Dagne Geremew, Assefa Ats-biha Tesfu, Philipp Slusallek, Thamar Solorio, and Dietrich Klakow. 2025. ProverbEval: Exploring LLM evaluation challenges for low-resource language understanding. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 6250–6266, Albuquerque, New Mexico. Association for Computational Linguistics. ↗
  9. Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S. Khan, Salman Khan, and Rao M. Anwer. 2024. Camel-bench: A comprehensive arabic lmm benchmark. Preprint, arXiv:2410.18976. ↗
  10. R.W. Gibbs. 1994. The Poetics of Mind: Figurative Thought, Language, and Understanding. The Poetics of Mind: Figurative Thought, Language, and Understanding. Cambridge University Press. find
  11. Girma Yohannis Bade, Jose Luis Oropeza, and Olga Kolesnikova. 2026. Pragmatic generalization in llms: Insights from fine-tuning and evaluating on multilingual sarcasm. In Advances in Soft Computing, pages 218–230, Cham. Springer Nature Switzerland. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783. ↗
  13. M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham Abdullah Alyahya, Sultan AlRashed, Faisal Abdulrahman Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman Alsubaihi, Maryam Al Mansour, Saad Amin Hassan, Dr. Majed Alrubaian, Ali Alammari, Zaki Alawami, and 7 others. 2025. AL-Lam: Large language models for arabic and english. In The Thirteenth International Conference on Learning Representations. find
  14. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. ↗ our card
  15. Anubha Kabra, Emmy Liu, Simran Khanuja, Alham Fikri Aji, Genta Winata, Samuel Cahyawijaya, Anuoluwapo Aremu, Perez Ogayo, and Graham Neubig. 2023. Multi-lingual and multi-cultural figurative language understanding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8269–8284, Toronto, Canada. Association for Computational Linguistics. ↗
  16. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022a. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606. ↗ our card
  17. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022b. FLUTE: Figurative language understanding through textual explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7139–7159, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. our card
  18. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 388–395, Barcelona, Spain. Association for Computational Linguistics. our card
  19. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin. 2024. ArabicMMLU: Assessing massive multitask language understanding in Arabic. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5622–5640, Bangkok, Thailand. Association for Computational Linguistics. ↗ our card
  20. Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, and Yejin Choi. 2025. CulturalBench: A robust, diverse and challenging benchmark for measuring LMs’ cultural knowledge through human-AI red-teaming. In Proceedings of the 63rd Annual Meeting of the Association for Computational Zoltán Kövecses. 2005. Metaphor in Culture: Universality and Variation. Cambridge University Press, Cambridge. find
  21. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez-García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, and 3 others. 2025. Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages. Preprint, arXiv:2406.09948. ↗ our card
  22. Joshua Lee, Wyatt Fong, Alexander Le, Sur Shah, Kevin Han, and Kevin Zhu. 2025. Pragmatic metacognitive prompting improves LLM performance on sarcasm detection. In Proceedings of the 1st Workshop on Computational Humor (CHum), pages 63–70, Online. Association for Computational Linguistics. ↗
  23. Viet Thanh Pham, Zhuang Li, Lizhen Qu, and Gholamreza Haffari. 2025. CultureInstruct: Curating multi-cultural instructions at scale. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 9207–9228, Albuquerque, New Mexico. Association for Computational Linguistics. find
  24. Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. Culturellm: Incorporating cultural differences into large language models. In Advances in Neural Information Processing Systems, volume 37, pages 84799–84838. Curran Associates, Inc. our card
  25. Chen Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych. 2024. Are multilingual LLMs culturally-diverse reasoners? an investigation into multicultural proverbs and sayings. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2016–2039, Mexico City, Mexico. Association for Computational Linguistics. ↗
  26. Ben Prystawski, Paul Thibodeau, Christopher Potts, and Noah D. Goodman. 2023. Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models. Preprint, arXiv:2209.08141. ↗
  27. David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cueva, Fajri Koto, Fauzan Farooqui, and 57 others. 2024. Cvqa: Culturally-diverse multilingual visual question answering benchmark. Preprint, arXiv:2406.05967. ↗
  28. Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the ability of language models to interpret figurative language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4437–4452, Seattle, United States. Association for Computational Linguistics. ↗ our card
  29. Abdelrahman Sadallah, Junior Cedric Tonga, Khalid Almubarak, Saeed Almheiri, Farah Atif, Chatrine Qwaider, Karima Kadaoui, Sara Shatnawi, Yaser Alesh, and Fajri Koto. 2025. Commonsense reasoning in arab culture. Preprint, arXiv:2502.12788. ↗
  30. Samar Mohamed Magdy, Sang Yun Kwon, Fakhraddin Alwajih, Safaa Taher Abdelfadil, Shady Shehata, and Muhammad Abdul-Mageed. 2025. JAWAHER: A multidialectal dataset of Arabic proverbs for LLM benchmarking. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 12320–12341, Albuquerque, New Mexico. Association for Computational Linguistics. ↗
  31. Farzad Sharifian. 2015. Cultural linguistics, 1st edition, pages 473–492. Routledge, United Kingdom. find
  32. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025. Fanar: An arabic-centric multimodal generative ai platform. Preprint, arXiv:2501.13944. ↗ our card
  33. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157. ↗
  34. W. Mieder. 1993. Proverbs are Never Out of Season: Popular Wisdom in the Modern Age. Oxford University Press. find
  35. Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Amandeep Kumar, Kartik Kuckreja, Mykola Maslych, Wafa Al Ghallabi, Mihail Mihaylov, Chao Qin, Abdelrahman M Shaker, Mike Zhang, Mahardika Krisna Ihsani, Amiel Esplana, Monil Gokani, and 50 others. 2025. All languages matter: Evaluating lmms on culturally diverse 100 languages. Preprint, arXiv:2411.16508. ↗
  36. W. Mieder. 2004. Proverbs: A Handbook. Greenwood Folklore Handbooks. Bloomsbury Academic. Basel Mousi, Nadir Durrani, Fatema Ahmad, Md. find
  37. Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, and Firoj Alam. 2024. Aradice: Benchmarks for dialectal and cultural capabilities in llms. Preprint, arXiv:2409.11404. ↗ our card
  38. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. You are tasked with selecting the correct answer to the following question. Choose the correct answer from the options provided. Only output the letter corresponding to the correct answer and nothing else. Question: [QUESTION] Options: A. [OPTION 1] B. [OPTION 2] C. [OPTION 3] Answer: Arnav Yayavaram, Siddharth Yayavaram, Prajna Devi Upadhyay, and Apurba Das. 2024. BERT-based idiom identification using language translation and word cohesion. In Proceedings of the Joint Workshop on Multiword Expressions and Universal Dependencies (MWE-UD) @ LREC-COLING 2024, pages 220– 230, Torino, Italia. ELRA and ICCL. ↗ our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attia-2026-figurative-cultural-knowledge-llms-investigating/