📄 Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change

Summary

The paper asks whether distributional methods can tell apart two causes of meaning change. One is cultural, where new technologies or social usage give a word a new sense. The other is linguistic, where regular processes such as subjectification or grammaticalization gradually reshape a word. The authors use the diachronic SGNS (word2vec) embeddings from their HistWords project. These are aligned across decades and built from four Google N-gram corpora (English All, English Fiction, French, German; 1800–1990) and from COHA (1850–2000, word forms and lemmas).

They compare two measures. The standard global measure is the cosine distance between a word's vectors in consecutive decades. Their new local neighborhood measure tracks how a word's similarity to its nearest neighbors changes. Linear mixed-model regressions show that the local measure attributes relatively more change to nouns, while the global measure favors verbs. Since nouns tend to shift for cultural reasons and verbs through regular drift, the authors argue that each measure suits a different task. Comparing the two can also suggest which kind of change a word went through.

Key points

• Local neighborhood measure:
  • For each of two decades, take the word's k nearest neighbors and merge the two lists.
  • For each decade, build a "second-order" vector of the word's cosine similarities to every word in the merged set.
  • Score change as the cosine distance between the two second-order vectors.
  • The experiments use k = 25, and results stayed consistent for k between 10 and 50.
• Regression design:
  • Each dataset gets two regressions, one per measure, with change scores normalized to zero mean and unit variance.
  • Predictors are word frequency, decade (as a category) and noun-vs-verb; per-word random effects handle correlation over time.
  • Only words in the top 10,000 by frequency are included, proper nouns are excluded, and words occurring fewer than 500 times in COHA are dropped.
• Dataset sizes: the numbers of nouns and verbs tested vary widely by dataset. For example, Google English All has 5299 nouns and 2722 verbs, while COHA (Lemma) has 3389 nouns and 783 verbs.
• Main result: in all six datasets, the local measure assigns relatively higher change to nouns than the global measure does. The pattern still holds when adjectives and adverbs are grouped with verbs.
• Case studies (English Google, 1850s vs 1990s):
  • actually, must and promise (subjectification, deontic-to-epistemic modality, performative speech acts) score higher on the global measure.
  • gay, virus and cell (cultural or technological shifts) score higher on the local measure.

Interesting ideas

• I1. Neighbor-restricted second-order vectors: describing a word by its similarities to a small, merged set of neighbors, rather than the whole vocabulary, makes the comparison focus on its closest semantic relations. It also makes the score less sensitive to drift of the entire embedding space. This could help any cross-period or cross-corpus comparison where alignment is imperfect.
• I2. Disagreement as a signal: the difference between the global and local scores can serve as a rough indicator of the type of change, not only its size. The same trick could separate, for instance, topical from stylistic shifts in other domains.
• I3. Validation through a linguistic proxy: with no gold labels for "cultural" versus "linguistic" change, the authors rely on the known tendencies of nouns versus verbs to test what each measure is sensitive to. This kind of indirect check is useful wherever labeled change data is scarce.
• I4. Controlled statistics for change rates: a mixed model that includes frequency and per-word random effects avoids mistaking frequency effects or repeated measurements for real differences. This is a sound default for any diachronic study of word-level change.
• I5. Matching the measure to the goal: the authors recommend the local measure for spotting cultural change points and the global one for studying grammaticalization. The general lesson is that a change metric should be chosen for the research question, not by habit.