Elamur Research⌕
☰
home / papers / 2026 / As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language

As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language

Paper · 2026 · Jasmine Owers, Edwin Simpson and Martha Lewis

Summary

The paper asks whether current language models can correctly read sentences that are both figurative and negated, for example a simile that conveys the opposite of its adjective. The authors focus on out-of-the-box use, without fine-tuning. They add new labels to the Fig-QA training set covering metaphor vs. simile, negating vs. intensifying similes (split into explicit "not" and antonym types), tense and concreteness. They also build a small literal counterpart set to separate the effect of negation from that of figurativeness. A range of models, including Llama models, is evaluated with three elicitation methods: embedding comparison, mid-phrase likelihood, and a question-answer prompt. An embedding-space analysis of one model is also reported.

Figurative negation is consistently harder than literal negation or figurative language on its own. Results also depend heavily on the prompting method. The question-answer setup scores highest but is arguably least like real use. This matters because practitioners often treat LLMs as robust interpreters of everyday language, and benchmark conclusions can change with evaluation format.

Key points

  • Fig-QA has 10,256 figurative sentences, each with one correct and one incorrect paraphrase. Its train split (9,110 items) was annotated as 1,940 metaphors and 7,170 similes, including 3,302 sentences in negation/intensification pairs.
  • Negating similes total 1,649 (1,029 "not" type, 620 antonym type). Intensifying similes total 1,653. A manual check covered 184 sentences (about 2%).
  • A new literal negation dataset contains 256 sentences matched to Fig-QA items. Accuracy on these literal items was never below accuracy on the figurative ones.
  • With the question-answer method, the newest models handled literal negation well but dropped on figurative negated similes. Tense showed a small effect, and concreteness showed no clear effect.
  • Code and data are available at github.com/jrdowers/Negation-and-Fig-Lang. The paper is published in TACL vol. 14 (2026) under CC-BY 4.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AmsterdamAmsterdam · Netherlands144 works · 173,906 citations🎓University of BristolBristol · United Kingdom14 works · 725 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksIdioms & multiword expressions · 217 works

Cite

Cite as Jasmine Owers, Edwin Simpson and Martha Lewis. 2026. As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.781

@article{owers2026as,
  title = {{As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language}},
  author = {Jasmine Owers, Edwin Simpson and Martha Lewis},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.781},
  url = {https://doi.org/10.1162/tacl.a.781}
}

References

35 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rodrigo Agerri. 2008. Metaphor in Textual Entailment. In Coling 2008: Companion volume: Posters, pages 3–6, Manchester, UK.
  2. Coling 2008 Organizing Committee. find
  3. Yuri Bizzoni and Shalom Lappin. 2018. Predicting Human Metaphor Paraphrase Judgments with Deep Neural Networks. In Proceedings of the Workshop on Figurative Language Processing, pages 45–55, New Orleans, Louisiana. Association for Computational Linguistics. find
  4. Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow. find
  5. Marc Brysbaert, Amy Warriner, and Victor Kuperman. 2013. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior Research Methods, 46. find
  6. Cristina Cacciari and Sam Glucksberg. 1994. Understanding figurative language. In Handbook of Psycholinguistics, pages 447–477. Academic Press, San Diego, CA, US. find
  7. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022a. It‘s not Rocket Science: Interpreting Figurative Language in Narratives. Transactions of the Association for Computational Linguistics, 10:589–606. Place: Cambridge, MA Publisher: MIT Press.
  8. Tuhin Chakrabarty, Debanjan Ghosh, Adam Poliak, and Smaranda Muresan. 2021. Figurative Language in Recognizing Textual Entailment. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3354–3361, Online. Association for Computational Linguistics.
  9. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022b. FLUTE: Figurative Language Understanding through Textual Explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7139– 7159, Abu Dhabi, United Arab Emirates. Associ…
  10. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680, Copenhagen,…
  11. Iker García-Ferrero, Begoña Altuna, Javier Alvez, Itziar Gonzalez-Dios, and German Rigau. 2023. This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8596–8615, Singapore.… find
  12. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  13. Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020. An Analysis of Natural Language Inference Benchmarks through the Lens of Negation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9106…
  14. Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, and Aaron Courville. 2021. Understanding by Understanding Not: Modeling Negation in Language Models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:…
  15. George Lakoff and Mark Johnson. 1980. Metaphors We Live By. University of Chicago Press. find
  16. Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the Ability of Language Models to Interpret Fgurative Language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4437–4452.… find
  17. AI @ Meta Llama Team. 2024. The Llama 3 Herd of Models. ArXiv:2407.21783v3 [cs.AI].
  18. Francois Meyer and Martha Lewis. 2020. Modelling Lexical Ambiguity with Density Matrices. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 276–290, Online. Association for Computational Linguistics. find
  19. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. ArXiv:1301.3781v3 [cs.CL].
  20. OpenAI. 2024a. GPT-4o System Card. ArXiv:2410.21276 [cs.CL].
  21. OpenAI. 2024b. OpenAI o1 System Card. ArXiv:2412.16720 [cs.CL].
  22. Nicholas Ichien, Dušan Stamenković, and Keith J. Holyoak. 2024. Large language model displays emergent ability to interpret novel literary metaphors. Metaphor and Symbol, 39(4):296– 309. find
  23. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543.
  24. Daniel Jurafsky and James H. Martin. 2025. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd edition. Online manuscript released January 12, 2025. find
  25. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  26. Nora Kassner and Hinrich Schütze. 2020. Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguis-Richard M. Roberts and Roger J. Kreuz. 1994. Why Do People Use Figurative Langua…
  27. Psychological Science, 5(3):159–163. lisher: SAGE Publications Inc. find
  28. Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Ope…
  29. Pub-Ekaterina Shutova. 2010. Automatic Metaphor Interpretation as a Paraphrasing Task. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 1029–1037. Association for Computational Linguistics. find
  30. Gerard J. Steen, Aletta G. Dorst, J. Berenike Herrmann, Anna A. Kaal, and Tina Krennmayr. 2010. Metaphor in usage. Cognitive Linguistics, 21(4):765–796. Publisher: De Gruyter Mouton Section: Cognitive Linguistics. find
  31. Kevin Stowe, Prasetya Utama, and Iryna Gurevych. 2022. IMPLI: Investigating NLI Models’ Performance on Figurative Language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5375–5388, Dublin, Ireland. Association for Computatio…
  32. Thinh Hung Truong, Timothy Baldwin, Karin Verspoor, and Trevor Cohn. 2023. Language Models are not Naysayers: an Analysis of Language Models on Negation Benchmarks. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 101–114, Toronto, Canada. Associa…
  33. Tony Veale, Ekaterina Shutova, and Beata Beigman Klebanov. 2016. Metaphor: A Computational Perspective. Synthesis Lectures on Human Language Technologies. Springer International Publishing, Cham. find
  34. Xiaoyu Tong, Rochelle Choenni, Martha Lewis, and Ekaterina Shutova. 2024. Metaphor Understanding Challenge Dataset for LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3517–3536, Bangkok, Thailand. Association for Computa…
  35. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/owers-2026-as-easy-as-rocket-science/