Elamur Research⌕
☰
home / papers / 2024 / From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

Paper · 2024 · Xenia Ohmer, Elia Bruni and Dieuwke Hupkes

Summary

The paper asks whether strong benchmark scores of text-trained LLMs reflect real grasp of meaning or just skill at producing plausible text. Drawing on Frege's sense/reference distinction and Wittgenstein's view of meaning as use, the authors argue that a model that truly understands should give the same answers when one meaning is expressed in different ways. They call this test multisense consistency. They study GPT-3.5 and have the model produce its own meaning-preserving variants: English paraphrases and translations into German, Italian, Dutch and Swedish. They then check whether its answers agree across these versions. Testing starts with simple factual questions and moves to four NLU benchmarks. GPT-3.5 is often inconsistent, even when it only has to restate a fact. Follow-up analyses suggest that its grasp of a task changes with the form in which the task is presented. The work offers an empirical, model-generated way to separate form from meaning. It also questions how far LLMs can serve as models of human understanding.

Key points

  • Object of study is GPT-3.5, evaluated on the original English (en), on rephrased English versions labeled en P, and on translated versions labeled de T, it T, nl T and sv T.

  • Meaning-preserving variants are generated by the model itself, so the test checks for a self-consistent notion of meaning rather than alignment with human judgments.

  • The controlled factual probes covered:

    • birth years of writers,
    • company headquarters,
    • winners of sports events.

    Answers often diverged across senses, and some variants produced refusals or full-sentence replies.

  • The NLU benchmarks used were PAWS, XNLI, COPA and Belebele.

  • Ablations translated or paraphrased only the instruction or only the input data. These were used to locate where inconsistencies come from, and they support a sense-dependent task understanding.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Osnabrück UniversityOsnabrück · Germany7 works · 250 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksInterpretability · 130 works

Cite

Cite as Xenia Ohmer, Elia Bruni and Dieuwke Hupkes. 2024. From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00529

@article{ohmer2024form,
  title = {{From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency}},
  author = {Xenia Ohmer, Elia Bruni and Dieuwke Hupkes},
  journal = {Computational Linguistics},
  year = {2024},
  doi = {10.1162/coli\_a\_00529},
  url = {https://doi.org/10.1162/coli_a_00529}
}

References

36 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdou, Mostafa, Kulmizev Artur Hershcovich Daniel Frank Stella, Ellie Pavlick and Anders Søgaard. 2021. Can language models encode perceptual structure without grounding? A case study in color. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 109–132. https://d…
  2. Arehalli, Suhas, Brian Dillon, and Tal Linzen. 2022. Syntactic surprisal from neural models predicts, but underestimates, human processing difficulty from syntactic ambiguities. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 301–313. https://doi.org/1…
  3. Bender, Emily M. and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198. https://doi.org/10.18653/v1/2020 .acl-main.463 Berglund, Lukas, Meg… ↗
  4. Biever, Celeste. 2023. ChatGPT broke the turing test—the race is on for new ways to assess AI. Nature (News Feature), 619:686–689. https://doi.org/10.1038/d41586-023-02361-7 Bisk, Yonatan, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jo… ↗
  5. Chen, Jifan, Eunsol Choi, and Greg Durrett. 2021. Can NLI models verify QA systems’ predictions? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3841–3854. https://doi.org/10 .18653/v1/2021.findings-emnlp.324 Christiansen, Morten H. and Nick Chater. 1999. Toward a con… find
  6. Frank, Stefan and Rens Bod. 2011. Insensitivity of the human sentence-processing system to hierarchical structure. Psychological Science, 22(6):829–834. https://doi.org/10 .1177/0956797611409589, PubMed: 21586764 Frege, Gottlob. 1892. Über Sinn und Bedeutung [“On sense and reference”]. Zeitschrift… find
  7. Futrell, Richard and Roger Levy. 2017. Noisy-context surprisal as a human sentence processing cost model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 688–698. https://doi.org/10.18653/v1/E17-1065 Gentne… ↗
  8. Hernandez, A., P. Li, and B. Macwhinney. 2005. The emergence of competing modules in bilingualism. Trends in Cognitive Sciences, 9(5):220–225. https://doi.org/10.1016/j.tics.2005.03.003, PubMed: 15866148 Hosseini, Arian, Siva Reddy, Dzmitry Bahdanau, R. Devon Hjelm, Alessandro Sordoni, and Aaron Co… — [doi:10.1016/j.tics.2005.03.003](https://doi.org/10.1016/j.tics.2005.03.003) ↗
  9. Huang, Kuan Jung, Suhas Arehalli, Mari Kugemoto, Christian Muxica, Grusha Prasad, Brian Dillon, and Tal Linzen. 2023. Surprisal does not explain syntactic disambiguation difficulty: Evidence from a large-scale benchmark. PsyArXiv Preprint, z38u6. https://doi.org/10.31234/osf .io/z38u6 Hupkes, Dieuwk… — [doi:10.31234/osf](https://doi.org/10.31234/osf) ↗
  10. Hupkes, Dieuwke, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske… find
  11. Jang, Myeongjun, Deuk Sin Kwon, and Thomas Lukasiewicz. 2022. BECEL: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3680–3696. find
  12. Jang, Myeongjun and Thomas Lukasiewicz. 2023. Consistency analysis of ChatGPT. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15970–15985. https://doi.org/10.18653/v1/2023.emnlp -main.991 Johnson-Laird, Philip N. and Marco Ragni. 2023. What should r… ↗
  13. Jumelet, Jaap, Milica Denic, Jakub Szymanik, Dieuwke Hupkes, and Shane Steinert-Threlkeld. 2021. Language models use monotonicity to assess NPI licensing. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4958–4969. https://doi.org /10.18653/v1/2021.findings-acl.43… ↗
  14. Lakretz, Yair, Dieuwke Hupkes, Alessandra Vergallito, Marco Marelli, Marco Baroni, and Stanislas Dehaene. 2021. Mechanisms for handling nested dependencies in neural-network language models and humans. Cognition, 213:104699. Special Issue in Honour of Jacques Mehler, Cognition’s founding editor. htt…
  15. Liang, Yaobo, Nan Duan, Yeyun Gong, Wu Ning, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Wu Winnie, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and M… — XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
  16. Linzen, Tal and Marco Baroni. 2021. Syntactic structure from deep learning. Annual Review of Linguistics, 7:195–212. https://doi.org/10.1146/annurev-linguistics -032020-051035 Linzen, Tal, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencie… ↗
  17. Liu, Yunzhe, Raymond J. Dolan, Zeb Kurth-Nelson, and Timothy E. J. Behrens. 2019. Human replay spontaneously reorganizes experience. Cell, 178(3):640–652. https://doi.org/10 .1016/j.cell.2019.06.012, PubMed: 31280961 Mahowald, Kyle, Anna A. Ivanova, Idan A. Blank, Nancy Kanwisher, Joshua B. Tenenbau… find
  18. McCoy, R., Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. 2023. Thomas Embers of autoregression: Understanding large language models through the problem they are trained to solve. ArXiv preprint, arXiv:2309.13638. https://doi.org/10.1073/pnas .2322420121 McCoy, Tom, Ellie Pavlick,… ↗
  19. Mitchell, Eric, Joseph Noh, Siyan Li, Will Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, and Manning Christopher. 2022. Enhancing self-consistency and performance of pre-trained language models through natural language inference. In Proceedings of the 2022 Conference on Empirical Methods in… find
  20. Nie, Yixin, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885–4901. https://doi.org/10.18653/v1/20… ↗
  21. Ouyang, Long, Wu Jeffrey, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. T… — Training language models to follow instructions with human feedback
  22. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318. https://doi.org/10.3115/1073083 .1073135 Patel, Roma and Pavl… ↗
  23. Pavlick, Ellie. 2023. Symbols and grounding in large language models. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 381(2251):20220041. https://doi.org/10.1098/rsta.2022.0041, PubMed: 37271171 Piantadosi, Steven. 2023. Modern language models ref… ↗
  24. Piantadosi, Steven and Felix Hill. 2022. Meaning without reference in large language models. In NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI), pages 1–8. Podkorytov, Maksim, Daniel Biś, and Xiuwen Liu. 2021. How can the [mask] know? The sources and limitations of knowledge in BERT. I…
  25. Ray Choudhury, Sagnik, Anna Rogers, and Isabelle Augenstein. 2022. Machine reading, fast and slow: When do models “understand” language? In Proceedings of the 29th International Conference on Computational Linguistics, pages 78–93. Rei, Ricardo, José G. C. Souza, Duarte Alves, Chrysoula Zerva, Ana…
  26. Roemmele, Melissa, Cosmin Adrian Bejan, and Andrew S. Gordon. 2011. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Proceedings of the 2011 AAAI Spring Symposium Series, pages 90–95. find
  27. Ruder, Sebastian, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Dan Garrette, Graham Neubig, and Melvin Johnson. 2021. XTREME-R: Towards more challenging and nuanced multilingual evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Na…
  28. Tenenbaum, Joshua B., Charles Kemp, Thomas L. Griffiths, and Noah D. Goodman. 2011. How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022):1279–1285. https://doi.org/10.1126/science.1192788, PubMed: 21393536 Timkey, William and Tal Linzen. 2023. A language model with limited… ↗
  29. Ulmer, Dennis, Dieuwke Hupkes, and Elia Bruni. 2019. Assessing incrementality in sequence-to-sequence models. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 209–217. https://doi.org/10.18653/v1/W19-4324 ↗
  30. Vaidya, Avinash R., Henry M. Jones, Johanny Castillo, and David Badre. 2021. Neural representation of abstract task structure during generalization. eLife, 10:e63226. https://doi.org/10.7554/eLife.63226, PubMed: 33729156 Van Schijndel, Marten and Tal Linzen. 2018. Modeling garden path effects withou… — [doi:10.7554/elife.63226](https://doi.org/10.7554/elife.63226) ↗
  31. Van Schijndel, Marten and Tal Linzen. 2021. Single-stage prediction models do not explain the magnitude of syntactic disambiguation difficulty. Cognitive Science, 45(6):e12988. https://doi.org/10.1111/cogs.12988, PubMed: 34170031 Wang, Alex, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julia… ↗
  32. Wang, Alex, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355…
  33. Wang, Haohan, Sun Da, and Eric P. Xing. 2019. What if we simply swap the two text fragments? A straightforward yet effective way to test the robustness of methods to confounding signals in natural language inference tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33,… — [doi:10.1609/aaai.v33i01.33017136](https://doi.org/10.1609/aaai.v33i01.33017136) find
  34. Wang, Xiaozhi, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Li Juanzi. 2022. Finding skill neurons in pre-trained transformer-based language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11132–11152. https://doi.org/10.18653/v1/2022… ↗
  35. Wittgenstein, Ludwig. 1953. Philosophical investigations. find
  36. Philosophische Untersuchungen. Macmillan. Yang, Yinfei, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joi…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ohmer-2024-form-s-meaning-probing-semantic/