Elamur Research⌕
☰
home / papers / 2024 / From cart to truck: meaning shift through words in English in the last two centuries

From cart to truck: meaning shift through words in English in the last two centuries

Paper · 2024 · Esteban Rodriguez Betancourt and Edgar Casasola Murillo

Summary

Most computational work on semantic change follows a single word and asks how its meaning drifts. This paper turns the question around and takes the onomasiological view: for a fixed concept, which word expressed it in each period? The authors use the pre-trained decade-level word2vec skip-gram embeddings of Hamilton et al. (2016), "All English (1800s-1900s)", built from Google N-Grams. They delete words whose vectors are all zeros. Each decade is then rotated into the 1990s space with orthogonal Procrustes. For a modern word's vector, Gensim's similar_by_vector returns the nearest words in every earlier decade. They examine energy, transport, entertainment and computing terms. The neighbours often line up with plausible historical predecessors. The modern vector for truck lands nearest cart in the 1800s, aircraft lands near ship, and petroleum and diesel land near coal and steam. The study is qualitative, with no gold standard. The authors present the method as a way into past viewpoints and stress that domain experts must interpret the results.

Key points

  • Data: HistWords-style embeddings, one space per decade from the 1800s to the 1990s, trained on the Google Books N-Gram corpus. The authors note this corpus leans toward scientific writing.
  • Alignment: every decade is aligned to a single anchor, the 1990s, using orthogonal Procrustes. Neighbours are retrieved as the top-N words by vector similarity in Gensim.
  • Concepts studied (14):
    • energy: petroleum, diesel, electricity, nuclear
    • transport: bus, truck, train, aircraft
    • entertainment: radio, cinema, television
    • computing: computer, internet, email
  • Notable findings:
    • Nuclear clusters with war and weapon vocabulary.
    • Computer sits near mathematical and laboratory terms until the 1940s, when it starts matching itself.
    • Email moves from gazette and courier, to telegram and phone, to telex and fax.
    • In the 1960s email has an unexplained neighbour, penguin.
  • Limitations named by the authors: one corpus, one language, coarse decade bins, and static rather than contextual embeddings.

Interesting ideas

  • I1. Reverse lookup across time. Instead of asking what an old word meant, take a modern concept vector and search an older space for its nearest word, which surfaces historical stand-ins like cart for truck.
  • I2. One shared reference frame. Rotating every period into a single anchor space with an orthogonal (distance-preserving) map makes all comparisons happen in common coordinates without distorting each space.
  • I3. Clean degenerate vectors first. Zero-vector entries are dropped before any neighbour search, because they would otherwise produce meaningless matches.
  • I4. Treat odd neighbours as hypotheses, not facts. Pairs like email–penguin or nationalism–religious groups are flagged for expert judgement rather than reported as findings. This is a sound policy for any output mined from embeddings.
  • I5. Separate corpus artifacts from meaning change. The switch from theatres to theater after the 1890s probably reflects more American text in the corpus, not a change in the concept.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): this could map archaic vocabulary in fables and old tales onto modern ontology nodes (e.g. a wagon or carriage linked to vehicle), so the world model reasons over familiar classes.
  • I2 (back): the same anchoring trick could align lexical spaces of different languages to one pivot, supporting translation through the meta-world.
  • I3 (learn): add a cheap pre-filter for empty or degenerate features in silver data, so they cannot quietly pollute similarity-based voting or repair-rule induction.
  • I4 (learn): embedding-derived links could be kept in a "needs review" tier and checked against curated knowledge or negative controls before entering the knowledge graph.
  • I5 (tokens): normalising British/American spelling variants to one lemma would keep downstream statistics from splitting a single concept in two.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Universidad de Costa Rica.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Esteban Rodriguez Betancourt and Edgar Casasola Murillo. 2024. From cart to truck: meaning shift through words in English in the last two centuries. arXiv:2408.16209.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2408.16209 · arXiv 2408.16209

@misc{betancourt2024cart,
  title = {{From cart to truck: meaning shift through words in English in the last two centuries}},
  author = {Esteban Rodriguez Betancourt and Edgar Casasola Murillo},
  year = {2024},
  doi = {10.48550/arxiv.2408.16209},
  eprint = {2408.16209},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.16209}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 4356–4364, Red Hook, NY, USA. Curran Associates Inc. ↗
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  3. Joachim Grzega and Marion Schöner. 2007. English and General Historical Lexicology : Materials for Onomasiology Seminars. Katholische Universität Eichstätt. find
  4. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489–1501, Berlin, Germany. Association for Computational Linguistics. ↗
  5. Simon Hengchen, Nina Tahmasebi, Dominik Schlechtweg, and Haim Dubossarsky. 2021. Challenges for Computational Lexical Semantic Change. ↗
  6. Andrey Kutuzov, Lilja Øvrelid, Terrence Szymanski, and Erik Velldal. 2018. Diachronic word embeddings and semantic shifts: a survey. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1384–1397, Santa Fe, New Mexico, USA. Association for Computational Linguistics. ↗
  7. Andrey Kutuzov, Erik Velldal, and Lilja Øvrelid. 2019. One-to-X analogical reasoning on word embeddings: a case for diachronic armed conflict prediction from news texts. In Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change, pages 196–201, Florence, Italy. Association for Computational Linguistics. find
  8. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 1188–1196, Bejing, China. PMLR. ↗
  9. Tomas Mikolov, Kai Chen, G. Corrado, and J. Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of 1st International Conference on Learning Representations, {ICLR} 2013. ↗
  10. Stefano Montanelli and Francesco Periti. 2023. A survey on contextualised semantic shift detection. find
  11. Radim Rehurek and Petr Sojka. 2011. Gensim–python framework for vector space modelling. NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic, 3(2). find
  12. Peter H. Schönemann. 1966. A generalized solution of the orthogonal Procrustes problem. j-PSYCHO, 31(1):1–10. find
  13. Terrence Szymanski. 2017. Temporal word analogies: Identifying lexical replacement with diachronic word embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 448–453, Vancouver, Canada. Association for Computational Linguistics. find
  14. Nina Tahmasebi, Lars Borin, and Adam Jatowt. 2019. Survey of computational approaches to lexical semantic change. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/betancourt-2024-cart-truck-meaning-shift-through/