Elamur Research⌕
☰
home / papers / 2021 / Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data

Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data

Paper · 2021 · Roxana Girju and David Peng

Summary

The paper asks how English text encodes sensory experience. It focuses on the most frequent perception verbs and splits them along two axes. The first is an Agentive vs. Experiential contrast: look/see, listen/hear and touch/feel. The second is the five sense modalities, with smell and taste standing alone because they lack such a split. The authors took sentences containing these verbs from the POS-tagged UMBC WebBase corpus. For each verb they collected frequent content-word lemmas found close to it and scored those words with an association index adapted from olfactory research. They then grouped the descriptors using word embeddings and clustering, and compared the resulting spaces with the Lancaster Sensorimotor Norms. The main qualitative finding is a difference in stance. Experiential verbs (see, hear) sit among words of distant, reasoned knowing. Agentive verbs (look, listen) sit among words of deliberate, close and intuitive engagement. The descriptor spaces also overlap across senses, especially vision with touch and taste with smell and touch. The work matters for understanding cross-modal meaning, and the authors see a use in building experiential analogies for blind people.

Key points

  • Source data is the UMBC WebBase collection, which holds over three billion words from a February 2007 crawl and is POS-tagged in Penn Treebank style. The "perception corpus" is roughly an eighth of it (the first 50 json files), lowercased and lemmatized.
  • Sentence counts per verb:
    • Vision: 642,256 in total (see 397,441, look 244,815).
    • Touch: 206,283 in total (feel 178,635, touch 27,648).
    • Hearing: 166,177 in total (hear 128,856, listen 37,321).
    • Taste and smell: only about 17.7k and 7.9k sentences.
  • Descriptor extraction works as follows:
    • Only content-word lemmas (nouns, verbs, adjectives, adverbs) count.
    • A word must appear at least 10 times within ±4 tokens of the verb.
    • Each verb keeps up to its 1,600 most frequent descriptors, a cutoff set by manual inspection.
    • Descriptors are ranked by an OAI-style log2 ratio of in-modality frequency to total frequency.
  • Lancaster Norms comparison: Interoception was added as a sixth dimension. About 49% of Lancaster's Vision words overlap with the Olfactory descriptors and 37% with the Gustatory descriptors.
  • Example observations:
    • Words like first-hand and rumor cluster in the auditory space, mirroring the contrast between "I see" (insight) and "I hear" (hearsay).
    • Taste leans toward smell and touch, driven by food words about texture and temperature.
  • Code and collocation data are released on GitHub (davidtpeng/sensory-spaces-perceptual-verbs).

Interesting ideas

  • I1. Separating perception verbs into agentive (deliberate act) and experiential (perception that just happens) pairs exposes a systematic meaning contrast, not just synonymy.
  • I2. A log2 ratio of a word's frequency in modality-specific contexts to its total frequency gives a cheap, interpretable score of how strongly that word belongs to a sense.
  • I3. Checking corpus-derived descriptor spaces against a human-rated norm set (Lancaster) is a way to validate an automatic lexical resource without manual annotation.
  • I4. The see/hear evidential contrast (insight vs. hearsay) shows that the perception verb itself signals how reliable the reported knowledge is.
  • I5. The authors note that verb frames differ across the divide (look at vs. see) and that syntax would better tune the context window than a flat ±4 tokens.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): VerbNet-based frames could mark look/listen/touch subjects as Agent and see/hear/feel subjects as Experiencer, so that deliberate attention is not confused with passive perception when interpreting who did what.
  • I2 (world): such a per-word sensory-association score could be compiled into the knowledge layer as a value attached to lexical entries, letting the world model tag entity properties as visual, tactile or olfactory with a traceable number.
  • I3 (learn): induced lexical classes or sensory tags could be scored against Lancaster norms as an external witness, with mismatch rates serving as an evaluation signal.
  • I4 (answer): when a fable or text reports that a character "heard" something rather than "saw" it, readers could lower the certainty of that fact and explain the answer by pointing to the verb.
  • I5 (tree): collecting descriptors through UD dependency arcs (obl, obj, xcomp) instead of linear windows would yield cleaner, explainable collocations for building lexical resources.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States108 works · 67,432 citations

📚 Shelves 4 Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Psychology · 798 works Experimental and Cognitive Psychology · 463 works

🧭 Topics 1 English · 1,373 works

Cite

Cite as Roxana Girju and David Peng. 2021. Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data. arXiv:2110.09721 · CEUR-WS.org 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2110.09721 · arXiv 2110.09721

@misc{girju2021exploring,
  title = {{Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data}},
  author = {Roxana Girju and David Peng},
  year = {2021},
  doi = {10.48550/arxiv.2110.09721},
  eprint = {2110.09721},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2110.09721}
}

References

25 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Farina, Embodied cognition: dimensions, domains and applications, Adaptive Behavior 29 (2021) 73–88. find
  2. M. M. Louwerse, P. Jeuniaux, The linguistic and embodied nature of conceptual processing, Cognition 114 (2010) 96–104. find
  3. B. Winter, Sensory linguistics: language, perception and metaphor, John Benjamins Publishing Company, Amsterdam; Philadelphia, 2019. find
  4. L. S. Roque, K. H. Kendrick, E. Norcliffe, P. Brown, R. Defina, M. Dingemanse, T. Dirksmeyer, N. Enfield, S. Floyd, J. Hammond, G. Rossi, S. Tufvesson, S. van Putten, A. Majid, Vision verbs dominate in conversation across cultures, but the ranking of non-visual verbs varies, Cognitive Linguistics 26 (2015) 31–60. find
  5. L. S. Roque, K. H. Kendrick, E. Norcliffe, A. Majid, Universal meaning extensions of perception verbs are grounded in interaction, Cognitive Linguistics 29 (2018) 371–406. find
  6. D. Divjak, Exploring the Grammar of Perception, Functions of Language 22 (2015) 44–68. find
  7. D. Peng, R. Girju, To feel: Spaces of sensory experience in natural language data, Affective Science 178 (2021) 37–49. find
  8. B. Levin, English verb classes and alternations, Chicago, 1993. find
  9. A. Viberg, The verbs of perception, A typological study. Linguistics 21 (1984) 123–162. find
  10. D. Lynott, L. Connell, M. Brysbaert, J. Brand, J. Carney, The Lancaster Sensorimotor Norms: multidimensional measures of perceptual and action strength for 40,000 English words, Behavior Research Methods 3 (2020) 1271 – 1291. find
  11. L. Han, A. L. Kashyap, T. Finin, J. Mayfield, J. Weese, UMBC EBIQUITY-CORE: Semantic Textual Similarity Systems, in: The 2nd Joint Conference on Lexical and Computational Semantics, ACL, 2013. our card
  12. M. Marcus, G. Kim, M. A. Marcinkiewicz, R. MacIntyre, A. Bies, M. Ferguson, K. Katz, B. Schasberger, The Penn Treebank: Annotating predicate argument structure, in: Workshop on Human Language Technology, 1994. our card
  13. G. Iatropoulos, P. Herman, A. Lansner, J. Karlgren, M. Larsson, J. K.Olofsson, The language of smell: Connecting linguistic and psychophysical properties of odor descriptors, Cognition 178 (2018) 37–49. find
  14. S. McGregor, B. McGillivray, A distributional semantic methodology for enhanced search in historical records: A case study on smell, in: A. Barbaresi, H. Biber, F. Neubarth, R. Osswald (Eds.), the 14th Conference on Natural Language Processing, KONVENS 2018, Vienna, Austria, Österreichische Akademie der Wissenschaften, 2018, pp. 1–11. find
  15. T. Hörberg, M. Larsson, J. Olofsson, Mapping the Semantic Organization of the English Odor Vocabulary Using Natural Language Data, in: PsyArXiv, volume 89, PsyArXiv, 2020. find
  16. T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, 2013. arXiv:1301.3781. ↗ our card
  17. L. Kaufman, P. J. Rousseeuw, Finding Groups in Data, John Wiley & Sons, New York, 1990. find
  18. H. J. Ward, Hierarchical grouping to optimize and objective function, Journal of American Statistical Association 58 (2018) 236 – 244. find
  19. P. J. Rousseeuw, Silhouettes: a graphical aid to the interpretation and validation of cluster analysis, Computational and Applied Mathematics 20 (1987) 53–65. find
  20. R. Tibshirani, W. Guenther, T. Hastie., Estimating the number of clusters in a data set via the gap statistic, Journal of the Royal Statistical Society Series B 63 (2001) 411–423. find
  21. G. W. Parrott, Emotions in social psychology: Essential readings, Psychology Press, W.G. Parrot (ed), 2001. find
  22. R. Plutchik, The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice, American scientist 89 (2001) 344–350. find
  23. I. Croijmans, A. Majid, Odor naming is difficult, even for wine and coffee experts, in: D. C. Noelle, R. Dale, A. S. Warlaumont, J. Yoshimi, T. Matlock, C. D. Jennings, P. P. Maglio (Eds.), Proceedings of the 37th Annual Meeting of the Cognitive Science Society, CogSci 2015, Pasadena, California, USA, 2015. find
  24. A. Majid, E. Gibson, T. M. Luhrmann, J. H. McDermott, A. Arshamian, Percepts and concepts across cultures, in: C. Kalish, M. A. Rau, X. J. Zhu, T. T. Rogers (Eds.), Proceedings of the 40th Annual Meeting of the Cognitive Science Society, Madison, WI, USA, 2018. find
  25. B. Landau, L. Gleitman, Language and experience. Evidence from the blind child, Harvard University Press, Cambridge, MA, 1985. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/girju-2021-exploring-sensory-spaces-english-perceptual/