Elamur Research⌕
☰
home / papers / 2025 / Exploring the Structure of AI-Induced Language Change in Scientific English

Exploring the Structure of AI-Induced Language Change in Scientific English

Paper · 2025 · Riley Galpin, Bryce Anderson and Tom S. Juzek

Summary

Since about 2022, words such as "delve", "crucial" and "intricate" have risen sharply in scientific abstracts, and this rise is usually blamed on LLMs. The authors ask how this shift is structured. Their null "replacement hypothesis" says a spiking word displaces its near-synonyms. To test it, they took 32 focal words from prior literature and keyed each by lemma and POS. They gathered thesaurus synonyms for each and used an LLM to keep the five most relevant to academic writing. Frequency changes from 2020 to 2024 were then measured in POS-tagged PubMed abstracts. The hypothesis mostly fails. Whole synonym clusters tend to rise together, which points to a pragmatic or stylistic shift rather than simple word swapping. The authors also ran an exploratory analysis of declining words, prompted by the drop in "important". Those declines look messier and closer to ordinary organic change than the abrupt spikes do. The work matters for understanding LLM-driven homogenization of scientific style and for treating such drift as a form of model bias.

Key points

  • Data: 28.8m PubMed abstracts from 1975–2024, about 6.94bn tokens.
    • Tagged with spaCy en_core_web_sm 3.8.0, which took roughly 140 hours on one desktop machine.
    • Punctuation, symbols, numbers, proper nouns and X tags were dropped.
  • Measures: percent change in occurrences per million (OPM) between 2020 and 2024, with significance checked by a chi-square contingency test in SciPy.
  • Synonym sets: drawn from the Merriam-Webster thesaurus per POS, then narrowed to five per focal word by GPT-4o-mini.
    • Focal words without a thesaurus entry for their POS were kept alone.
  • Example spikes: "meticulously ADV" +991.83%, "underscore VERB" +765.78%, "showcase VERB" +508.12%.
    • Synonyms often rose too, e.g. "notable" +263.69% and "emphasize" +180.97%.
    • "important ADJ" fell 23.43%.
  • Role of POS: the change sits almost entirely in verbs, adjectives and adverbs.
    • The noun readings ("underscore NOUN", "boast NOUN", "intricacy NOUN", "showcase NOUN") stay below 1 OPM and barely matter.
  • Resource: code, word lists and graphs are at github.com/fsu-nlp/structure-ai-lang-change.

Interesting ideas

  • I1. Frame the obvious explanation (one word simply replaces its neighbours) as an explicit null hypothesis, and test it at the level of the whole semantic cluster rather than single words. This guards against over-reading isolated frequency spikes.
  • I2. Count lemma+POS pairs instead of surface forms, so that "underscore" as verb and as noun, or "potential" as noun and adjective, are tracked separately. This removes ambiguity that otherwise hides where change really happens.
  • I3. Build candidate sets from a curated lexical resource, then use an LLM only as a narrow relevance filter for the domain, with a fixed and logged prompt. The resource keeps control of the inventory while the LLM trims noise like "surf" for "delve".
  • I4. Study what disappears, not only what spikes, against a long historical baseline that separates abrupt shifts from slow organic trends. Asymmetric patterns between rises and falls are informative in their own right.
  • I5. The authors note that words like "crucial" are not low-information in themselves; they become filler only when nothing concrete in the situation backs them up.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when LLM teacher tiers produce silver data, check whether whole families of phrasings drift together against the original corpus, not just individual tokens, so stylistic contamination is caught as a group effect.
  • I2 (tokens): keep all corpus frequency statistics in crate en keyed by lemma plus UPOS, so tagger and lemma statistics stay comparable across corpora and years.
  • I3 (world): derive near-synonym groups from WordNet, let an LLM teacher only rank or prune them for a register, and store that decision as provenance next to the compiled ontology entry.
  • I4 (learn): in evaluation, monitor constructions and frames that vanish from parser or generator output over training rounds, using older data as a negative control.
  • I5 (back): let generation emit evaluative words like "crucial" or "significant" only when the text's world model holds a supporting state or relation, which keeps output grounded and explainable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Florida State UniversityTallahassee · United States10 works · 4,835 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksHistorical linguistics & language change · 250 works

Cite

Cite as Riley Galpin, Bryce Anderson and Tom S. Juzek. 2025. Exploring the Structure of AI-Induced Language Change in Scientific English. arXiv:2506.21817 · The International FLAIRS Conference Proceedings (Vol. 38), 2025.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.32473/flairs.38.1.138958 · arXiv 2506.21817

@misc{galpin2025exploring,
  title = {{Exploring the Structure of AI-Induced Language Change in Scientific English}},
  author = {Riley Galpin, Bryce Anderson and Tom S. Juzek},
  year = {2025},
  doi = {10.32473/flairs.38.1.138958},
  eprint = {2506.21817},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.32473/flairs.38.1.138958}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aitchison, J. 2005. Language change. In Cobley, P., ed., The Routledge Companion to Semiotics and Linguistics. London: Routledge. chapter 7. find
  2. Alemohammad, S.; Casco-Rodriguez, J.; Luzi, L.; Humayun, A. I.; Babaei, H.; LeJeune, D.; Siahkoohi, A.; and Baraniuk, R. G. 2023. Self-consuming generative models go mad. arXiv preprint arXiv:2307.01850. ↗
  3. Altman, D. G., and Bland, J. M. 1995. Statistics notes: Absence of evidence is not evidence of absence. Bmj 311(7003):485. ↗
  4. Baron, N. S. 2010. Always on: Language in an online and mobile world. Oxford University Press. find
  5. Bizzoni, Y.; Degaetano-Ortlieb, S.; Fankhauser, P.; and Teich, E. 2020. Linguistic variation and change in 250 years of english scientific writing: A data-driven approach. Frontiers in Artificial Intelligence 3:73. ↗
  6. Briesch, M.; Sobania, D.; and Rothlauf, F. 2023. Large language models suffer from their own output: An analysis of the self-consuming training loop. arXiv preprint arXiv:2311.16822. ↗
  7. Bybee, J. L. 2006. From usage to grammar: The mind’s response to repetition. Language 82(4):711– 733. ↗
  8. Crystal, D. 2008. Txtng: The gr8 db8. OUP Oxford. find
  9. Degaetano-Ortlieb, S., and Teich, E. 2018. Using relative entropy for detection and analysis of periods of diachronic linguistic change. In Proceedings of the second joint SIGHUM workshop on computational linguistics for cultural heritage, social sciences, humanities and literature, 22–33. ↗
  10. Geng, M.; Chen, C.; Wu, Y.; Chen, D.; Wan, Y.; and Zhou, P. 2024. The impact of large language models in academia: from writing to speaking. arXiv preprint arXiv:2409.13686. ↗
  11. Gray, A. 2024. Chatgpt” contamination”: estimating the prevalence of llms in the scholarly literature. arXiv preprint arXiv:2403.16887. ↗
  12. Haslwanter, T. 2016. An introduction to statistics with python. With applications in the life sciences. Switzerland: Springer International Publishing. ↗
  13. Haspelmath, M. 2000. Why is grammaticalization irreversible? Linguistics 37:1043–68. ↗
  14. Hataya, R.; Bao, H.; and Arai, H. 2023. Will large-scale generative models corrupt future datasets? In Proceedings of the IEEE/CVF International Conference on Computer Vision, 20555–20565. find
  15. Heine, B. 2017. Grammaticalization. The handbook of historical linguistics 573–601. find
  16. Hopper, P. J., and Thompson, S. A. 1984. The discourse basis for lexical categories in universal grammar. Language 60(4):703–752. ↗
  17. Juzek, T. S., and Ward, Z. B. 2025. Why does chatgpt” delve” so much? exploring the sources of lexical overrepresentation in large language models. In Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025). find
  18. Kobak, D.; Márquez, R. G.; Horvát, E.-Á.; and Lause, J. 2024. Delving into chatgpt usage in academic writing through excess vocabulary. arXiv preprint arXiv:2406.07016. ↗
  19. Koppenburg, P. 2024. Tweet on 01 april 2024. https://x.com/PKoppenburg/status/1774757167045788010. Accessed: 2025-01-23. ↗
  20. Krielke, M.-P. 2024. Cross-linguistic dependency length minimization in scientific language: Syntactic complexity reduction in english and german in the late modern period. Languages in Contrast 24(1):133–163. find
  21. Labov, W. 1973. Sociolinguistic patterns. University of Pennsylvania press. find
  22. Liang, W.; Izzo, Z.; Zhang, Y.; Lepp, H.; Cao, H.; Zhao, X.; Chen, L.; Ye, H.; Liu, S.; Huang, Z.; et al. 2024a. Monitoring ai-modified content at scale: A case study on the impact of chatgpt on ai conference peer reviews. arXiv preprint arXiv:2403.07183. ↗
  23. Liang, W.; Zhang, Y.; Wu, Z.; Lepp, H.; Ji, W.; Zhao, X.; Cao, H.; Liu, S.; He, S.; Huang, Z.; et al. 2024b. Mapping the increasing use of llms in scientific papers. arXiv preprint arXiv:2404.01268. ↗
  24. Liu, J., and Bu, Y. 2024. Towards the relationship between aigc in manuscript writing and author profiles: evidence from preprints in llms. arXiv preprint arXiv:2404.15799. ↗
  25. Mahajan, S. 2023. Artificial intelligence and its impacts on the society. Contemporary Social Sciences 32(4):135–151. ↗
  26. Makridakis, S. 2017. The forthcoming artificial intelligence (ai) revolution: Its impact on society and firms. Futures 90:46–60. ↗
  27. Matsui, K. 2024. Delving into pubmed records: Some terms in medical writing have drastically changed after the arrival of chatgpt. medRxiv 2024–05. find
  28. Menzel, K. 2022. Medical discourse in late modern english: Insights from a multidisciplinary corpus of scientific journal articles. In Corpus pragmatic studies on the history of medical discourse. John Benjamins. 79–104. find
  29. Merriam-Webster. 2025. Merriam-Webster Thesaurus. https://www.merriam-webster.com/thesaurus. Accessed: 2025-01-11. ↗
  30. Milroy, J., and Milroy, L. 1985. Linguistic change, social network and speaker innovation1. Journal of linguistics 21(2):339–384. find
  31. Montani, I.; Honnibal, M.; Boyd, A.; Landeghem, S. V.; and Peters, H. 2023. explosion/spacy: v3.7.2: Fixes for apis and requirements. Version v3.7.2, Zenodo. find
  32. National Library of Medicine. 2024. PubMed Database. https://pubmed.ncbi.nlm.nih.gov/. Accessed: 2024-12-06. ↗
  33. Pachegowda, C. 2023. The global impact of ai-artificial intelligence: Recent advances and future directions, a review. arXiv preprint arXiv:2401.12223. ↗
  34. Shapira, P. 2024. Delving into ”delve”. Accessed: 2025-01-23. find
  35. Thomason, S. G., and Kaufman, T. 2023. Language contact, creolization, and genetic linguistics. Univ of California Press. ↗
  36. Virtanen, P.; Gommers, R.; Oliphant, T. E.; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J.; van der Walt, S. J.; Brett, M.; Wilson, J.; Millman, K. J.; Mayorov, N.; Nelson, A. R. J.; Jones, E.; Kern, R.; Larson, E.; Carey, C. J.; Polat, İ.; Feng, Y.; Moore, E. W.; VanderPlas, J.; Laxalde, D.; Perktold, J.; Cimrman, R.; Henriksen, I.; Quintero, E. A.; Harris, C. R.; Archibald, A. M.; Ribeiro, A. H.; Pedregosa, F.; van Mulbregt, P.; and SciPy 1.0 Contributors. 2020. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17:261–272.
  37. Winford, D. 2003. An introduction to contact linguistics. (No Title). find
  38. Yakura, H.; Lopez-Lopez, E.; Brinkmann, L.; Serna, I.; Gupta, P.; and Rahwan, I. 2024. Empirical evidence of large language model’s influence on human spoken communication. arXiv preprint arXiv:2409.01754. ↗
  39. Zhang, X.; Xiong, W.; Chen, L.; Zhou, T.; Huang, H.; and Zhang, T. 2024. From lists to emojis: How format bias affects model alignment. arXiv preprint arXiv:2409.11704. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/galpin-2025-exploring-structure-ai-induced-language/