Elamur Research⌕
☰
home / papers / 2026 / A framework for annotating and modelling intentions behind metaphor use

A framework for annotating and modelling intentions behind metaphor use

Summary

Metaphors often serve communicative goals, such as stirring emotions, persuading, or making abstract ideas easier to grasp. Prior research usually examined these goals one at a time, and NLP had no unified scheme for labelling why a speaker chose a metaphor. The authors reviewed the literature on metaphor and intention and built a taxonomy of 9 intention categories. They refined it in several rounds against real examples from the VU Amsterdam Metaphor Corpus (VUAMC). Using this taxonomy and an annotation procedure, they released the first dataset of metaphors labelled for intention, with up to three categories allowed per metaphor. They then tested GPT-4 Turbo and Llama2-Chat (13B and 70B) on picking a single intention for a metaphor in context, in zero-shot and few-shot settings. All models beat the random baseline, but performance stayed low. The best result, from GPT-4, was about 45% accuracy. Recognising metaphor intent therefore remains hard for LLMs. The resource can support better metaphor understanding and help models decide when figurative language is appropriate.

Key points

  • The taxonomy has 9 categories, including Lexicalized metaphor, Artistic use of metaphor, Visualization (a merger of "Vividness" and "Imageability") and Persuasiveness. The dataset is drawn from the VUAMC and is available at github.com/GMichelli/intentions-behind-metaphor.
  • Two of the authors annotated the data, and a reliability study gave encouraging results. The VUAMC consists mostly of conventional, indirect metaphors.
  • GPT-4 reached 43.30% average accuracy zero-shot and 45.09% with five examples. Llama2-70B-Chat peaked at 39.00% (9-shot), and Llama2-13B-Chat at 30.88% (9-shot with short prompts).
  • In few-shot prompts, removing the category explanations usually lowered accuracy. GPT-4 scored its highest F1 on Lexicalized metaphor and Visualization.
  • The models had to choose only one category, which simplifies a task that annotators could label with several intentions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.starsem-conference.4

@inproceedings{michelli2026framework,
  title = {{A framework for annotating and modelling intentions behind metaphor use}},
  author = {Michelli, Gianluca and Tong, Xiaoyu and Shutova, Ekaterina},
  booktitle = {Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.starsem-conference.4},
  url = {https://doi.org/10.18653/v1/2026.starsem-conference.4}
}

References

40 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Andrew Gargett and John Barnden. 2015. Modeling the interaction between sensory and affective meanings for detecting metaphor. In Proceedings of the Third Workshop on Metaphor in NLP, pages 21–30, Denver, Colorado. Association for Computational Linguistics. find
  2. G. E. M. Anscombe. 1957. Intention. Cambridge, Mass.: Harvard University Press. find
  3. Ron Artstein and Massimo Poesio. 2008. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. find
  4. Raymond W. Jr Gibbs. 1999. Intentions in the Experience of Meaning. Cambridge University Press. find
  5. OpenAI, :, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mo Bavarian, Jeff Belgum, Irwan Bello, Jake… find
  6. A. Goatly. 1997. The Language of Metaphors, 1 edition. Routledge. find
  7. Annette M. Green. 1997. Kappa statistics for multiple raters using categorical classifications. find
  8. H. Paul Grice. 1957. Meaning. Philosophical Review, 66(3):377–388. find
  9. Michael Haugh and Kasia M. Jaszczolt. 2012. Speaker intentions and intentionality. In Keith Allan and Kasia Jaszczolt, editors, Cambridge Handbook of Pragmatics, pages 87–112. Cambridge University Press. find
  10. M. B. Hesse. 1966. Models and Analogies in Science. Newman history and philosophy of science series. Ind. find
  11. Arthur M. Jacobs and Annette Kinder. 2017. “the brain is the prisoner of thought”: A machine-learning assisted quantitative narrative analysis of literary metaphors for use in neurocognitive poetics. Metaphor and Symbol, 32(3):139–160. find
  12. Lynn Katz Fainsilber and Andrew Ortony. 1987. Metaphorical uses of language in the expression of emotions. Metaphor and Symbol - METAPHOR SYMB, 2:239–250. find
  13. Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, Anna Kocoń, Bartłomiej Koptyra, Wiktoria Mieleszczenko-Kowszewicz, Piotr Miłkowski, Marcin Oleksy, Maciej Piasecki, Łukasz Radliński, Konr… find
  14. Klaus Krippendorff. 1980. Content Analysis: An Introduction to Methodology. Sage Publications, Inc., Beverly Hills, CA. find
  15. George Lakoff and Mark Johnson. 1980. Metaphors we Live by. University of Chicago Press, Chicago. find
  16. Ridwan Mahbub, Ifrad Khan, Samiha Anuva, Md Shihab Shahriar, Md Tahmid Rahman Laskar, and Sabbir Ahmed. 2023. Unveiling the essence of poetry: Introducing a comprehensive dataset and benchmark for poem summarization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc… find
  17. Andreas Musolff. 2004. Metaphor and political discourse: Analogical reasoning in debates about Europe. Palgrave Macmillan London. find
  18. Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan… find
  19. Paul Ricœur. 1975. La métaphore vive. Éditions du seuil, 27, rue Jacob, Paris VIe. find
  20. Richard M. Roberts and Roger J. Kreuz. 1994. Why do people use figurative language? Psychological Science, 5(3):159–163. find
  21. John R. Searle. 1983. Intentionality: An Essay in the Philosophy of Mind. Cambridge University Press. find
  22. P. Sopory and J. P. Dillard. 2002. The persuasive effects of metaphor: A meta-analysis. Human Communication Research, 28(3):382–419. find
  23. Rand Spiro, Paul J. Feltovich, Richard Coulson, and Daniel Anderson. 1989. Multiple analogies for complex concepts: Antidotes for analogy-induced misconception in advanced knowledge acquisition. In S. Vosniadou and A. Ortony, editors, Similarity and analogical reasoning, pages 498–530. Cambridge Uni… find
  24. G. J. Steen, A. G. Dorst, J. B. Herrmann, A. A. Kaal, and T. Krennmayr. 2010a. VU Amsterdam Metaphor Corpus. find
  25. E. Oring. 2003. Engaging Humor. University of Illinois Press. find
  26. Andrew Ortony. 1975. Why metaphors are necessary and not just nice1. Educational Theory, 25:45 – 53. find
  27. Gerard Steen. 2008. The paradox of metaphor: Why we need a three-dimensional model of metaphor. Metaphor and Symbol, 23(1):213–241. find
  28. Allan Paivio, John C. Yuille, and Stephen A. Madigan. 1968. Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology, 76:1–25. find
  29. Gerard Steen. 2011. The contemporary theory of metaphor - now new and improved! Review of Cognitive Linguistics, 9(1):26–64. find
  30. Rebecca Passonneau. 2006. Measuring agreement on set-valued items (MASI) for semantic and pragmatic annotation. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), Genoa, Italy. European Language Resources Association (ELRA). find our card
  31. Gerard Steen. 2014. Deliberate metaphor affords conscious metaphorical cognition. Cognitive Semiotics, 5(1-2):179–197. find
  32. Gerard Steen. 2017. Deliberate metaphor theory: Basic assumptions, main tenets, urgent issues. Intercultural Pragmatics, 14(1):1–24. find
  33. Rebecca J. Passonneau. 2004. Computing reliability for coreference annotation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA). find
  34. Gerard Steen. 2023. Thinking by metaphor, fast and slow: Deliberate metaphor theory offers a new model for metaphor and its comprehension. Frontiers in Psychology, 14. find
  35. Prisca Piccirilli and Sabine Schulte Im Walde. 2022. What drives the use of metaphorical language? negative insights from abstractness, affect, discourse coherence and contextualized word representations. In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 299–3… find
  36. Gerard Steen, Lettie Dorst, J. Herrmann, Anna Kaal, Tina Krennmayr, and Trijntje Pasma. 2010b. A method for linguistic metaphor identification: From MIP to MIPVU. John Benjamins. find
  37. Carina Rasse, Alexander Onysko, and Francesca M. M. Citron. 2020. Conceptual metaphors in poetry interpretation: a psycholinguistic approach. Language and Cognition, 12(2):310–342. find
  38. Xiaoyu Tong, Rochelle Choenni, Martha Lewis, and Ekaterina Shutova. 2024. Metaphor understanding challenge dataset for llms. find
  39. The BNC Baby. 2005. The BNC Baby, version 2. Distributed by Oxford University Computing Services on behalf of the BNC Consortium. find
  40. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Gudrun Reijnierse, Christian Burgers, Tina Krennmayr, and Gerard Steen. 2018. Dmip: A method for iden-A Bhosale, Dan Bikel, Lukas Blecher, Cristian Canto… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/michelli-2026-framework-annotating-modelling-intentions-behind/