Elamur Research⌕
☰
home / papers / 2026 / Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations

Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations

Summary

The paper asks whether LLMs organize vocabulary the way people do, using free word association as a probe. The authors follow the LWOW elicitation protocol, prompting Mistral-7B, Llama-3.1-8B and Qwen-2.5-32B 100 times per cue for three associates each, at four temperatures. They compare first responses (R1) with human R1 answers from SWOW-EN on a sample of 490 cues. Two experiments follow. The first compares how word frequency (from a Wikipedia dump) and concreteness (Brysbaert et al. norms) of responses relate to their cues. The second introduces metrics of response variability and typicality that go beyond plain human–model overlap.

On frequency and concreteness, all three models follow the human patterns. They diverge on diversity. Qwen acts like one stereotypical respondent, giving very typical answers with little spread. Mistral and Llama spread their answers more but match humans less. Raising the temperature increases variability and lowers typicality. The study suggests that lexicon-probing work should report and control model size and temperature.

Key points

  • Human baseline: SWOW-EN, with 12,282 cues answered by 100 participants giving three words each. The study uses a uniform sample of 500 cues, reduced to 490 after filtering, and checks that the sample is representative.
  • Model data: three open models at temperatures 0.3, 1, 1.5 and 2 yield 12 LLM datasets of 49K instances per response rank. The datasets are released at github.com/mariro8/LLMWA.
  • Model versions: Qwen is the quantized Qwen2.5-32B-Instruct-GPTQ-Int8. It replaces the closed Claude model used in LWOW, which also left its temperature unreported.
  • Lexical factors: frequency and concreteness are measured as response-to-cue ratios. Models reproduce human tendencies on both. Appendix ANOVAs show temperature effects with small effect sizes (η² ≤ .035), strongest for Mistral.
  • Stated limitations: only open-source models, a cue subset, purely quantitative metrics, and large variation across cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5fozb5xnyvio

@inproceedings{rodriguez2026modeling,
  title = {{Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations}},
  author = {Rodriguez, Maria A. and Candito, Marie and Huyghe, Richard},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5fozb5xnyvio},
  url = {https://doi.org/10.63317/5fozb5xnyvio}
}

References

21 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Katherine Abramski, Riccardo Improta, Giulio Rossetti, and Massimo Stella. 2025. The “llm world of words” english free association norms generated by large language models. Scientific data, 12(1):803. find
  2. Katherine Abramski, Clara Lavorati, Giulio Rossetti, and Massimo Stella. 2024. "llm-generated word association norms". In Proceedings of the Third International Conference on Hybrid Human-Artificial Intelligence (HHAI), pages 3–13, Malmö, Sweden. find
  3. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. find
  4. Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2014. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior research methods, 46(3):904–911. find
  5. P. Cramer. 1968. Word Association. Word Association. Academic Press. find
  6. Simon De Deyne, Danielle J Navarro, Amy Perfors, Marc Brysbaert, and Gert Storms. 2019. The “small world of words” english word association norms for over 12,000 cue words. Behavior research methods, 51:987–1006. find
  7. Simon De Deyne and Gert Storms. 2008. Word associations: Network and semantic properties. Behavior research methods, 40(1):213–231. find
  8. Annette M de Groot. 1989. Representational aspects of word imageability and word frequency as assessed through word association. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(5):824. find
  9. Tess Fitzpatrick. 2007. Word association patterns: unpacking the assumptions. International Journal of Applied Linguistics, 17:319–331. find
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, et al. 2024. The llama 3 herd of models. find
  11. Felix Hill, Anna Korhonen, and Christian Bentz. 2014. A quantitative empirical analysis of the abstract/concrete distinction. Cognitive Science, 38(1):162–177. find
  12. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  13. Paul Meara. 1983. Word associations in a foreign language. Nottingham Linguistics Circular, 11(2):29–38. find
  14. Douglas L Nelson and Cathy L McEvoy. 2000. What is this think called frequency? Memory & Cognition, 28(4):509–522. find
  15. Douglas L Nelson and Thomas A Schreiber. 1992. Word concreteness and word structure as independent determinants of recall. Journal of memory and language, 31(2):237–260. find
  16. David S Palermo and James J Jenkins. 1964. Word association norms: Grade school through college. find
  17. David C Rubin and Michael Friendly. 1986. Predicting which words get recalled: Measures of free recall, availability, goodness, emotionality, and pronunciability for 925 nouns. Memory & cognition, 14(1):79–94. find
  18. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  19. Špela Vintar, Mojca Brglez, and Aleš Žagar. 2024. How human-like are word associations in generative models? an experiment in slovene. In Proceedings of the Workshop on Cognitive Aspects of the Lexicon@ LREC-COLING 2024, pages 42–48. find
  20. Andreas Waldis, Yotam Perlitz, Leshem Choshen, Yufang Hou, and Iryna Gurevych. 2024. Holmes: A benchmark to assess the linguistic competence of language models. find
  21. Bei Xiao, Xufeng Duan, David A. Haslett, and Zhenguang Cai. 2025. Human-likeness of LLMs in the mental lexicon. In to appear in the proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL 2025). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rodriguez-2026-modeling-human-lexicon-under-temperature/