Elamur Research⌕
☰
home / papers / 2024 / Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models

Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models

Paper · 2024 · Carina Kauf, Emmanuele Chersoni, Alessandro Lenci et al.

Summary

The paper asks whether explicit prompting or log probabilities (LogProbs) give a better measure of how much language models know about semantic plausibility, which serves as a proxy for world knowledge. The authors compare both approaches in three ~7B-parameter LMs, each tested in a base and an instruction-tuned version. Experiment 1 uses about 2,000 single-sentence minimal pairs from EventsAdapt and DTFit. Here, LogProbs-based preferences are contrasted with three zero-shot prompts: choosing between two sentences, giving a Yes/No verdict, and rating on a 1–7 scale. Experiment 2 uses short two-sentence scenarios from Jouravlev et al. (2019) to test whether a preceding context changes plausibility estimates. Three new context-sensitivity metrics are introduced for this purpose. Across both settings, LogProbs proved the more dependable signal, while prompting gave inconsistent and often weak results. Instruction tuning rarely changed how LogProbs track plausibility and sometimes reduced their agreement with humans. This matters because many newer models no longer expose probabilities. Researchers who rely on prompting alone may therefore underestimate what these models implicitly encode.

Key points

  • Data for Experiment 1:
    • EventsAdapt (Fedorenko et al., 2020) has 391 items with active/passive variants. It covers both impossible and unlikely implausibility, with animate–inanimate and animate–animate participants.
    • DTFit (Vassallo et al., 2018) adds animate-agent/inanimate-patient pairs, such as an actor winning an award versus winning a battle.
  • Prompting falls short: In both base and Instruct models, zero-shot prompting was less reliable than LogProbs for plausibility.
  • Effect of instruction tuning: LogProbs-derived estimates stayed largely unchanged after instruction tuning. In some cases they matched human judgments less well than the base model's did.
  • Context effects (Experiment 2):
    • At the target word, LogProbs reached roughly 90% accuracy against the ground truth and closely reproduced human sensibility-judgment patterns.
    • At the sentence level, models seem to recover after an unexpected word. As a result, anomalous and contextually licensed sentences become harder to tell apart.
  • Scope and resources: The study covers English only and models up to 7B parameters. Code and data are released at github.com/carina-kauf/llm-plaus-prob.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations🎓Hong Kong Polytechnic UniversityHong Kong · China84 works · 4,956 citations🎓University of PisaPisa · Italy25 works · 3,842 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Carina Kauf, Emmanuele Chersoni, Alessandro Lenci, Evelina Fedorenko and Anna A. Ivanova. 2024. Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models. arXiv:2403.14859.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2403.14859 · arXiv 2403.14859

@misc{kauf2024log,
  title = {{Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models}},
  author = {Carina Kauf, Emmanuele Chersoni, Alessandro Lenci, Evelina Fedorenko and Anna A. Ivanova},
  year = {2024},
  doi = {10.48550/arxiv.2403.14859},
  eprint = {2403.14859},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2403.14859}
}

References

51 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Laura Aina and Tal Linzen. 2021. The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty through Generation. In Proceedings of the EMNLP BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks. find
  2. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. 2023. The Falcon Series of Open Language Models. arXiv preprint arXiv:2311.16867.
  3. Khai Loong Aw, Syrielle Montariol, Badr AlKhamissi, Martin Schrimpf, and Antoine Bosselut. 2023. Instruction-tuning Aligns LLMs to the Human Brain. arXiv preprint arXiv:2312.00575. — [arXiv:2312.00575](https://arxiv.org/abs/2312.00575) find
  4. Dougla Bates. 2014. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823.
  5. Klinton Bicknell, Jeffrey L Elman, Mary Hare, Ken McRae, and Marta Kutas. 2010. Effects of Event Knowledge in Processing Verbal Arguments. Journal of Memory and Language, 63(4):489–505. find
  6. Terra Blevins, Hila Gonen, and Luke Zettlemoyer. 2023. Prompting Language Models for Linguistic Structure. In Proceedings of ACL. find
  7. Yongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang, Nicholas Roy, and Chuchu Fan. 2024. Prompt Optimization in Multi-step Tasks (PROMST): Integrating Human Feedback and Preference Alignment. arXiv preprint arXiv:2402.08702. — [arXiv:2402.08702](https://arxiv.org/abs/2402.08702) find
  8. Emmanuele Chersoni, Philippe Blache, and Alessandro Lenci. 2016. Towards a Distributional Model of Semantic Complexity. In Proceedings of the COLING Workshop on Computational Linguistics for Linguistic Complexity. find
  9. Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache, and Chu-Ren Huang. 2021. Not All Arguments Are Processed Equally: A Distributional Model of Argument Complexity. Language Resources and Evaluation, pages 1–28. find
  10. Emmanuele Chersoni, Enrico Santus, Ludovica Pannitto, Alessandro Lenci, Philippe Blache, and Chu-Ren Huang. 2019. A Structured Distributional Model of Sentence Meaning and Processing. Natural Language Engineering, 25(4):483–502. find
  11. Yew Ken Chia, Pengfei Hong, Lidong Bing, and Soujanya Poria. 2023. INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models. arXiv preprint arXiv:2306.04757.
  12. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling Instruction-finetuned Language Models. arXiv preprint arXiv:2210.11416. find
  13. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Goldberg. 2022. Measuring Causal Effects of Data Statistics on Language Model’s Factual Predictions. arXiv preprint arXiv:2207.14251. — [arXiv:2207.14251](https://arxiv.org/abs/2207.14251) find
  14. Evelina Fedorenko, Idan Asher Blank, Matthew Siegelman, and Zachary Mineroff. 2020. Lack of Selectivity for Syntax Relative to Word Meanings Throughout the Language Network. Cognition, 203:104348. find
  15. Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State. In Proceedings of NAACL.
  16. Jonathan Gordon and Benjamin Van Durme. 2013. Reporting Bias and Knowledge Acquisition. In Proceedings of the Workshop on Automated Knowledge Base Construction. find
  17. Michael Hanna, Yonatan Belinkov, and Sandro Pezzelle. 2023. When Language Models Fall in Love: Animacy Processing in Transformer Language Models. In Proceedings of EMNLP. find
  18. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right. In Proceedings of EMNLP.
  19. Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger P Levy. 2020. A Systematic Assessment of Syntactic Generalization in Neural Language Models. In Proceedings of ACL. find
  20. Jennifer Hu and Roger Levy. 2023. Prompting Is Not a Substitute for Probability Measurements in Large Language Models. In Proceedings of EMNLP.
  21. Jennifer Hu, Kyle Mahowald, Gary Lupyan, Anna Ivanova, and Roger Levy. 2024. Language Models Align with Human Judgments on key Grammatical Constructions. arXiv preprint arXiv:2402.01676. — [arXiv:2402.01676](https://arxiv.org/abs/2402.01676) find
  22. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. find
  23. Olessia Jouravlev, Rachael Schwartz, Dima Ayyash, Zachary Mineroff, Edward Gibson, and Evelina Fedorenko. 2019. Tracking Colisteners’ Knowledge States during Language Comprehension. Psychological Science, 30(1):3–19. find
  24. Cheongwoong Kang and Jaesik Choi. 2023. Impact of Co-occurrence on Factual Knowledge of Large Language Models. In Findings of EMNLP. find
  25. Nora Kassner and Hinrich Schütze. 2020. Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, but Cannot Fly. In Proceedings of ACL.
  26. Carina Kauf and Anna Ivanova. 2023. A Better Way to Do Masked Language Model Scoring. In Proceedings of ACL.
  27. Carina Kauf, Anna A Ivanova, Giulia Rambelli, Emmanuele Chersoni, Jingyuan Selena She, Zawad Chowdhury, Evelina Fedorenko, and Alessandro Lenci. 2023. Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely. Cognitive Science, 47(11):e13386. find
  28. Tatsuki Kuribayashi, Yohei Oseki, and Timothy Baldwin. 2024. Psychometric Predictive Power of Large Language Models. In Findings of NAACL. find
  29. Andrew Kyle Lampinen. 2022. Can Language Models Handle Recursively Nested Grammatical Structures? A Case Study on Comparing Models and Humans. arXiv preprint arXiv:2210.15303.
  30. Alessandro Lenci. 2011. Composing and Updating Verb Argument Expectations: A Distributional Semantic Model. In Proceedings of the ACL Workshop on Cognitive Modeling and Computational Linguistics. find
  31. Belinda Z Li, Maxwell Nye, and Jacob Andreas. 2021. Implicit Representations of Meaning in Neural Language Models. In Proceedings of ACL. find
  32. Jiaoda Li, Ryan Cotterell, and Mrinmaya Sachan. 2022. Probing via Prompting. In Proceedings of NAACL. find
  33. James A Michaelov and Benjamin K Bergen. 2020. How Well Does Surprisal Explain N400 Amplitude Under Different Experimental Conditions? In Proceedings of CONLL. find
  34. MosaicML NLP Team. 2023. Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs. www.mosaicml.com/blog/mpt-7b. find
  35. Mante S Nieuwland and Jos JA Van Berkum. 2006. When Peanuts Fall in Love: N400 Evidence for the Power of Discourse. Journal of Cognitive Neuroscience, 18(7):1098–1111. find
  36. Isabel Papadimitriou, Richard Futrell, and Kyle Mahowald. 2022. When Classifying Grammatical Role, BERT Doesn’t Care about Word Order... Except When It Matters. In Proceedings of ACL. find
  37. Paolo Pedinotti, Giulia Rambelli, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, and Philippe Blache. 2021. Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge. In Proceedings of *SEM. find
  38. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic Prompt Optimization with "Gradient Descent" and Beam Search. In Proceedings of EMNLP.
  39. Guanghui Qin and Jason Eisner. 2021. Learning how to Ask: Querying LMs with Mixtures of Soft Prompts. In Proceedings of NAACL.
  40. Shirley-Ann Rueschemeyer, Tom Gardner, and Cat Stoner. 2015. The Social N400 Effect: How the Presence of Other Listeners Affects Language Comprehension. Psychonomic Bulletin & Review, 22:128– 134. find
  41. Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. 2020. Masked language model scoring. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2699–2712, Online. Association for Computational Linguistics.
  42. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting. arXiv preprint arXiv:2310.11324. find
  43. Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, and Adina Williams. 2022. Language Model Acceptability Judgements Are Not Always Robust to Context. arXiv preprint arXiv:2212.08979. find
  44. James A Michaelov, Seana Coulson, and Benjamin K Bergen. 2023. Can Peanuts Fall in Love with Distributional Semantics? In Proceedings of CogSci. find
  45. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. In Proceedings of EMNLP. find
  46. Kanishka Misra, Allyson Ettinger, and Kyle Mahowald. 2024. Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently. arXiv preprint arXiv:2401.06640. find
  47. Ottokar Tilk, Vera Demberg, Asad Sayeed, Dietrich Klakow, and Stefan Thater. 2016. Event Participant Modelling with Neural Networks. In Proceedings of EMNLP. find
  48. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Grave Edouard, and Guillaume Lample. 2023. Llama: Open and Efficient Foundation Language Models. arXiv pr…
  49. Paolo Vassallo, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, and Philippe Blache. 2018. Event Knowledge in Sentence Processing: A New Dataset for the Evaluation of Argument Typicality. In Proceedings of the LREC Workshop on Linguistic and Neuro-Cognitive Resources (LiNCR). find
  50. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R Bowman. 2020. BLiMP: The Benchmark of Linguistic Minimal Pairs for English. Transactions of the Association for Computational Linguistics, 8:377– 392.
  51. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction Tuning for Large Language Models: A Survey. arXiv preprint arXiv:2308.10792.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kauf-2024-log-probabilities-reliable-estimate-semantic/