Elamur Research⌕
☰
home / papers / 2026 / Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models

Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models

Paper · 2026 · Yang Liu, Hongming Li, Melissa Xiaohui Qin et al.

Summary

SemanticQA is an English benchmark that tests how well language models handle semantic phrases, also called multiword expressions (MWE). These are word combinations whose meaning cannot be fully built from their parts. The authors argue that mainstream LM benchmarks focus on math, code and logic and largely ignore meaning below the sentence level. They also note that earlier MWE evaluations each used one phrase type and one task format.

SemanticQA brings existing MWE datasets together into one testbed with four phenomena: lexical collocations, idiomatic expressions, noun compounds and verbal constructions. Each phenomenon is probed through three atomic operations (classification, extraction and interpretation) and through chained sequences of these operations. Fixed prompt templates keep comparisons across models fair.

Models of many sizes and architectures do well on open-ended interpretation, which is scored with lenient similarity metrics. They do much worse on strict span extraction. In chained setups, mistakes made early carry through to later steps. The authors conclude that judging phrase-level semantics requires several tasks and metrics, and that scores from a single task can overstate how robust a model really is.

Key points

  • Coverage and tasks: The benchmark spans 10 tasks across four phrase types. Sources include Tedeschi et al. (2022) for idiom extraction, Fisas et al. (2020) for collocation extraction, Savary et al. (2023) for VMWE extraction and Coil and Shwartz (2023) for noun-compound interpretation.
  • Metrics: Classification and extraction are scored with accuracy, plus sequence-level accuracy for extraction. Interpretation is scored with METEOR, ROUGE-L and BERTScore. Chained tasks report both conditional accuracy (given a correct extraction) and overall end-to-end accuracy.
  • Oracle schema for VMWE extraction: Giving models an explicit semantic description of the target construction improves VMWE extraction. For DeepSeek-R1 in the 0-shot setting, accuracy rises from 51.6% to 64.1%. GPT-5, OpenAI o3 and Kimi-K2-Instruct also improve.
  • Classification difficulty: Accuracy falls sharply as the label set grows from 4 to 8 to 16 classes. For GPT-5 in one setting, accuracy drops from 91.3 to 63.4. End-to-end accuracy is far below conditional accuracy, which shows how errors cascade through the pipeline.
  • Limitations and availability: The benchmark is English-only and omits rarer MWE types such as multiword named entities. The code and data are available at github.com/jacklanda/SemanticQA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Beijing Institute for General Artificial IntelligenceBeijing · China7 works · 576 citations🎓University of Science and Technology BeijingBeijing · China8 works · 315 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 worksEvaluation & benchmarks · 1,220 worksIdioms & multiword expressions · 217 works

Cite

Cite as Yang Liu, Hongming Li, Melissa Xiaohui Qin, Chao Huang and Qiankun Liu. 2026. Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.210

@inproceedings{liu2026revisiting,
  title = {{Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models}},
  author = {Yang Liu, Hongming Li, Melissa Xiaohui Qin, Chao Huang and Qiankun Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.210},
  url = {https://doi.org/10.18653/v1/2026.acl-long.210}
}

References

88 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Lisa Alazraki, Lihu Chen, Ana Brassard, Joe Stacey, Hossein A. Rahmani, and Marek Rei. 2025. Agent-coma: A compositional benchmark mixing commonsense and mathematical reasoning in real-world scenarios. Preprint, arXiv:2508.19988. — [arXiv:2508.19988](https://arxiv.org/abs/2508.19988) find
  3. Shengnan An, Xunliang Cai, Xuezhi Cao, Xiaoyu Li, Yehao Lin, Junlin Liu, Xinxuan Lv, Dan Ma, Xuanlin Wang, Ziwen Wang, and Shuang Zhou. 2025. Amo-bench: Large language models still struggle in high school math competitions. Preprint, arXiv:2510.26768. find
  4. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. In Anthropic Blog. find
  5. Anthropic. 2025. Anthropic. https://www.anthropic.com/news/claude-sonnet-4-5. September 30, 2025. ↗
  6. Yuki Arase and Jun’ichi Tsujii. 2020. Compositional phrase alignment and beyond. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1611–1623, Online. Association for Computational Linguistics. find
  7. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. Program synthesis with large language models. ArXiv, abs/2108.07732.
  8. Mislav Balunovic, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. 2025. Matharena: Evaluating LLMs on uncontaminated math competitions. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. find
  9. Ekaba Bisong. 2019. Google colaboratory. Building machine learning and deep learning models on google cloud platform: a comprehensive guide for beginners, pages 59–64. find
  10. Lars Buijtelaar and Sandro Pezzelle. 2023. A psycholinguistic analysis of BERT’s representations of compounds. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2230–2241, Dubrovnik, Croatia. Association for Computational Linguistic… find
  11. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022a. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606.
  12. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022b. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606.
  13. I-Hsuan Chen, Yunfei Long, Qin Lu, and Chu-Ren Huang. 2017. Leveraging eventive information for better metaphor detection and classification. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 36–46, Vancouver, Canada. Association for Computational L… find
  14. Albert Coil and Vered Shwartz. 2023. From chocolate bunny to chocolate crocodile: Do language models understand noun compounds? In Findings of the Association for Computational Linguistics: ACL 2023, pages 2698–2710, Toronto, Canada. Association for Computational Linguistics. find
  15. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  16. Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke Van Der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017a. Multiword expression processing: A survey. Computational Linguistics, 43(4):837–892. find
  17. Mathieu Constant, Gülşen Eryiǧit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017b. Survey: Multiword expression processing: A Survey. Computational Linguistics, 43(4):837–892.
  18. DeepSeek. 2025. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645:633–638. find
  19. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, Maryland, USA. Association for Computational Linguistics.
  20. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  21. Luis Espinosa-Anke, Joan Codina-Filba, and Leo Wanner. 2021. Evaluating language models for the retrieval and categorization of lexical collocations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1406–1417, Online.… find
  22. Luis Espinosa-Anke, Steven Schockaert, and Leo Wanner. 2019. Collocation classification with unsupervised relation vectors. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5765–5772, Florence, Italy. Association for Computational Linguistics. find
  23. Luis Espinosa-Anke, Alexander Shvets, Alireza Mohammadshahi, James Henderson, and Leo Wanner. 2022. Multilingual extraction and categorization of lexical collocations with graph-aware transformers. In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 89–100, Seat… find
  24. Afsaneh Fazly, Paul Cook, and Suzanne Stevenson. 2009. Unsupervised type and token identification of idiomatic expressions. Computational Linguistics, 35(1):61–103.
  25. Beatriz Fisas, Luis Espinosa-Anke, Joan Codina-Filbá, and Leo Wanner. 2020. CollFrEn: Rich bilingual English–French collocation resource. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 1–12, online. Association for Computational Linguistics. find
  26. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021. Assessing the representations of idiomaticity in vector models with a noun compound dataset labeled at type and token levels. In Proceedings of the 59th Annual Meeting of the Association for Computatio… find
  27. Alexander Gelbukh and 1 others. 2012. Semantic analysis of verbal collocations with lexical functions, volume 414. Springer. find
  28. Tayyar Madabushi Harish, Gow-Smith Edward, Scarton Carolina, and Villavicencio Aline. 2021. AStitchIn-LanguageModels: Dataset and methods for the exploration of idiomaticity in pre-trained language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3464–3477, Pun… find
  29. Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, and Mor Geva. 2023. Understanding transformer memorization recall through idioms. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 248–264, Dubrovnik, Croatia. Assoc…
  30. Iris Hendrickx, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Stan Szpakowicz, and Tony Veale. 2013. SemEval-2013 task 4: Free paraphrases of noun compounds. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop… find
  31. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. In International Conference on Learning Representations.
  32. Xuming Hu, Junzhe Chen, Xiaochuan Li, Yufei Guo, Lijie Wen, Philip S. Yu, and Zhijiang Guo. 2024. Towards understanding factual knowledge of large language models. In The Twelfth International Conference on Learning Representations. find
  33. Sirui Huang, Yanggan Gu, Zhonghao Li, Xuming Hu, Li Qing, and Guandong Xu. 2025. StructFact: Reasoning factual knowledge from structured data with large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7521–7552, Vienna, Austria. Association for Computat… find
  34. Gemma Team Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram’e, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-Bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  35. Kimi. 2025. Kimi k2: Open agentic intelligence. Preprint, arXiv:2507.20534.
  36. Filip Klubička, Vasudevan Nedumpozhimana, and John Kelleher. 2023. Idioms, probing and dangerous things: Towards structural probing for idiomaticity in vector space. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 45–57, Dubrovnik, Croatia. Association for Computation… find
  37. Olga Kolesnikova. 2020. Automatic detection of lexical functions in context. Computación y sistemas, 24(3):1337–1352. find
  38. Keshav Kolluru, Gabriel Stanovsky, and Mausam. 2022. “covid vaccine is against covid but Oxford vaccine is made at Oxford!” semantic interpretation of proper noun compounds. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10407–10420, Abu Dhabi, Unite… find
  39. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  40. Jia Li, Ge Li, Xuanming Zhang, Yunfei Zhao, Yihong Dong, Zhi Jin, Binhua Li, Fei Huang, and Yongbin Li. 2024. Evocodebench: An evolving code generation benchmark with domain-specific evaluations. In Advances in Neural Information Processing Systems, volume 37, pages 57619–57641. Curran Associates, I… find
  41. Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, and Zilong Zheng. 2025. ReflectEvo: Improving meta introspection of small LLMs by learning self-reflection. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16948–16966, Vi… find
  42. Caroline Pasquer, Agata Savary, Carlos Ramisch, and Jean-Yves Antoine. 2020. Verbal multiword expression identification: Do we need a sledgehammer to crack a nut? In Proceedings of the 28th International Conference on Computational Linguistics, pages 3333–3345, Barcelona, Spain (Online). Internation… find
  43. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  44. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, and 1 others. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  45. Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, and Kai Chen. 2024. MathBench: Evaluating the theory and application proficiency of LLMs with a hierarchical mathematics benchmark. In Findings of the Association for Computationa… find
  46. Yang Liu, Jiaqi Li, and Zilong Zheng. 2026a. Rulereasoner: Reinforced rule-based reasoning via domain-aware dynamic sampling. In The Fourteenth International Conference on Learning Representations. find
  47. Yang Liu, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li, and Lingyong Yan. 2026b. LM-lexicon: Improving definition modeling via harmonizing semantic experts. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages… find
  48. Thang Luong, Dawsen Hwang, Hoang H Nguyen, Golnaz Ghiasi, Yuri Chervonyi, Insuk Seo, Junsu Kim, Garrett Bingham, Jonathan Lee, Swaroop Mishra, Alex Zhai, Huiyi Hu, Henryk Michalewski, Jimin Kim, Jeonghyun Ahn, Junhwi Bae, Xingyou Song, Trieu Hoang Trinh, Quoc V Le, and Junehyuk Jung. 2025. Towards r…
  49. Igor A. Mel’čuk. 1998. Collocations and lexical functions. Phraseology. Theory, analysis, and applications, pages 23–53. find
  50. Igor A. Mel’čuk. 2023. General phraseology: Theory and practice. John Benjamins. find
  51. Filip Miletić and Sabine Schulte im Walde. 2024. Semantics of multiword expressions in transformer-based models: A survey. Transactions of the Association for Computational Linguistics, 12:593–612.
  52. Geoffrey Nunberg, Ivan A. Sag, and Thomas Wasow. 1994. Idioms. Language, 70(3):491–538. find
  53. OpenAI. 2025. Openai. https://openai.com/ index/introducing-o3-and-o4-mini. April 16, 2025. ↗
  54. Thang Pham, Seunghyun Yoon, Trung Bui, and Anh Nguyen. 2023. PiC: A phrase-in-context dataset for phrase understanding and semantic search. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1–26, Dubrovnik, Croatia. Association for… find
  55. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI. Accessed: 2024-11-15. find
  56. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
  57. Parikshit Ram, Tim Klinger, and Alexander G. Gray. 2024. What makes models compositional? a theoretical view. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI ’24. find
  58. Carlos Ramisch. 2023. Multiword expressions in computational linguistics. Habilitation à diriger des recherches, Aix Marseille Université (AMU). find
  59. Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk, Marie Candito, Ashwini Vaidya, Verginica Barbu Mititelu, Archna Bhatia, Uxoa Iñurrieta, Voula Giouli, Tunga Güngör, Menghan Jiang, Timm Lichte, Chaya Liebeskind, Johanna Monti, Renata Ramisch, Sara Stymne, Abigail Walsh, and Hongzhi Xu.… — Edition 1.2 of the PARSEME Shared Task on Semi-supervised Identification of Verbal Multiword Expressions
  60. Carlos Ramisch, Abigail Walsh, Thomas Blanchard, and Shiva Taslimipoor. 2023a. A survey of mwe identification experiments: The devil is in the details. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 106–120. find
  61. Carlos Ramisch, Abigail Walsh, Thomas Blanchard, and Shiva Taslimipoor. 2023b. A survey of MWE identification experiments: The devil is in the details. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 106–120, Dubrovnik, Croatia. Association for Computational Linguistic… find
  62. María A Barrios Rodríguez. 2003. The domain of the lexical functions fact0, causfact0 and real1. learning, page 64. find
  63. Ivan A Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002a. Multiword expressions: A pain in the neck for nlp. In Computational Linguistics and Intelligent Text Processing: Third International Conference, CICLing 2002 Mexico City, Mexico, February 17–23, 2002 Proceedings 3,…
  64. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002b. Multiword expressions: A pain in the neck for nlp. In Computational Linguistics and Intelligent Text Processing, pages 1–15, Berlin, Heidelberg. Springer Berlin Heidelberg.
  65. Manfred Sailer and Stella Markantonatou. 2018. Multiword expressions: Insights from a multi-lingual perspective. Language Science Press. find
  66. Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. 2020. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947. find
  67. Agata Savary, Cherifa Ben Khelil, Carlos Ramisch, Voula Giouli, Verginica Barbu Mititelu, Najet Hadj Mohamed, Cvetana Krstev, Chaya Liebeskind, Hongzhi Xu, Sara Stymne, Tunga Güngör, Thomas Pickard, Bruno Guillaume, Eduard Bejček, Archna Bhatia, Marie Candito, Polona Gantar, Uxoa Iñurrieta, Albert… find
  68. Agata Savary, Carlos Ramisch, Silvio Cordeiro, Federico Sangati, Veronika Vincze, Behrang QasemiZadeh, Marie Candito, Fabienne Cap, Voula Giouli, Ivelina Stoyanova, and Antoine Doucet. 2017. The PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the 13…
  69. Alexander Shvets and Leo Wanner. 2022. The relation dimension in the identification and classification of lexically restricted word co-occurrences in text corpora. Mathematics, 10(20). find
  70. Vered Shwartz and Ido Dagan. 2019. Still a pain in the neck: Evaluating text representations on lexical composition. Transactions of the Association for Computational Linguistics, 7:403–419.
  71. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, and 1 others. 2025. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267.
  72. Giorgos Spathas and Dimitris Michelioudakis. 2021. States in the decomposition of verbal predicates. Natural Language & Linguistic Theory, 39(4):1253– 1306. find
  73. Joshua Tanner and Jacob Hoffman. 2023. MWE as WSD: Solving multiword expression identification with word sense disambiguation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 181–193, Singapore. Association for Computational Linguistics.
  74. Simone Tedeschi, Federico Martelli, and Roberto Navigli. 2022. ID10M: Idiom identification in 10 languages. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2715–2726, Seattle, United States. Association for Computational Linguistics.
  75. Christine Thielen. 1999. Review of "turning a bilingual dictionary into a lexical-semantic database" by thierry fontenelle. max niemeyer verlag 1997. Comput. Linguist., 25(3):447–449. find
  76. Stephen Tratz and Eduard Hovy. 2010. A taxonomy, dataset, and classifier for automatic noun compound interpretation. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 678–687, Uppsala, Sweden. Association for Computational Linguistics. find
  77. Robert Vacareanu, Marco A. Valenzuela-Escárcega, Rebecca Sharp, and Mihai Surdeanu. 2020. An unsupervised method for learning representations of multi-word expressions for semantic classification. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3346–3356, Barc… find
  78. Takashi Wada, Yuji Matsumoto, Timothy Baldwin, and Jey Han Lau. 2023. Unsupervised paraphrasing of multiword expressions. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4732–4746, Toronto, Canada. Association for Computational Linguistics. find
  79. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transf…
  80. Alison Wray. 2002. Formulaic language and the lexicon, volume 20. Cambridge University Press Cambridge. find
  81. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  82. Qianyu Yang, Yang Liu, Jiaqi Li, Jun Bai, Hao Chen, Kaiyuan Chen, Tiliang Duan, Jiayun Dong, Xiaobo Hu, Zixia Jia, and 1 others. 2026. $onemillion-bench: How far are language agents from human experts? arXiv preprint arXiv:2603.07980. — [arXiv:2603.07980](https://arxiv.org/abs/2603.07980) find
  83. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. 2024. Natural language reasoning, a survey. ACM Comput. Surv., 56(12). find
  84. Ziheng Zeng and Suma Bhat. 2022. Getting BART to ride the idiomatic train: Learning to represent idiomatic expressions. Transactions of the Association for Computational Linguistics, 10:1120–1137.
  85. Ziheng Zeng, Kellen Cheng, Srihari Nanniyur, Jianing Zhou, and Suma Bhat. 2023. IEKG: A commonsense knowledge graph for idiomatic expressions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14243–14264, Singapore. Association for Computational Lingui…
  86. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find
  87. Jianing Zhou, Hongyu Gong, and Suma Bhat. 2021. PIE: A parallel idiomatic expression corpus for idiomatic sentence generation and paraphrasing. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 33–48, Online. Association for Computational Linguistics. find
  88. Jianing Zhou, Ziheng Zeng, Hongyu Gong, and Suma Bhat. 2022. Idiomatic expression paraphrasing without strong supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11774–11782. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-revisiting-pain-neck-semantic-reasoning/