Elamur Research⌕
☰
home / papers / 2024 / Mapping the Increasing Use of LLMs in Scientific Papers

Mapping the Increasing Use of LLMs in Scientific Papers

Paper · 2024 · Weixin Liang, Yaohui Zhang, Zhengxuan Wu et al.

Summary

Since ChatGPT appeared, many have guessed how often researchers use LLMs to write papers, but there has been no reliable measurement. Detectors that classify single documents are unreliable and can be biased against non-native writers. The authors take a different approach. They extend the distributional GPT quantification framework of Liang et al. (2024), which estimates the share of LLM-modified sentences across a whole corpus without labelling any individual text. It does this by fitting a mixture of word-occurrence distributions from human and LLM text using maximum likelihood. To build realistic LLM reference text, they had an LLM condense human-written paragraphs into outlines and then rewrite full paragraphs from those outlines. They applied the method to abstracts and introductions from 950,965 papers published from 2020 to early 2024. LLM-modified content rose sharply about five months after ChatGPT's release, with Computer Science growing fastest. The study offers a scalable way to monitor AI influence on scholarly writing. It also points to possible pressures behind that influence, such as competition and the push to publish quickly.

Key points

  • The corpus covers January 2020 to February 2024. It includes 773,147 arXiv papers (CS, EESS, Mathematics, Physics, Statistics), 161,280 bioRxiv papers and 16,538 articles from 15 Nature portfolio journals.
  • By February 2024, the estimated fraction (α) of LLM-modified sentences in arXiv Computer Science reached 17.5% in abstracts and 15.3% in introductions. Mathematics stayed lowest at 4.9% and 3.5%, and the Nature portfolio reached about 6.3% and 6.4%.
  • Words that LLMs favour, such as "realm", "intricate", "showcasing" and "pivotal", were rare in arXiv CS abstracts for over a decade. Their use jumped abruptly in 2023.
  • Higher α correlates with three features:
    • first authors who post preprints frequently,
    • papers in crowded areas, measured by embedding distance to the closest neighbouring paper,
    • shorter papers. CS papers under about 5,000 words reached 17.7% versus 13.6% for longer ones.
  • Estimates for 2022 stayed near zero, suggesting few false positives. The analysis targets ChatGPT only, and the reported associations are correlational rather than causal.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States506 works · 330,052 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksDecision Sciences · 249 worksInformation Systems and Management · 52 worksStatistics, Probability and Uncertainty · 65 worksHistory and Philosophy of Science · 62 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D Manning and James Y. Zou. 2024. Mapping the Increasing Use of LLMs in Scientific Papers. arXiv:2404.01268.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2404.01268 · arXiv 2404.01268

@misc{liang2024mapping,
  title = {{Mapping the Increasing Use of LLMs in Scientific Papers}},
  author = {Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D Manning and James Y. Zou},
  year = {2024},
  doi = {10.48550/arxiv.2404.01268},
  eprint = {2404.01268},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.01268}
}

References

79 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Scott Aaronson. Simons Institute Talk on Watermarking of Large Language Models, 2023. URL https://simons.berkeley.edu/talks/scott-aaronson-ut-austin-openai-2023-08-17. ↗
  2. Mikhail J. Atallah, Victor Raskin, Michael Crogan, Christian F. Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. Natural Language Watermarking: Design, Analysis, and a Proof-of-Concept Implementation. In Information Hiding, 2001. find
  3. Sameer Badaskar, Sachin Agarwal, and Shilpa Arora. Identifying Real or Fake Articles: Towards better Language Modeling. In International Joint Conference on Natural Language Processing, 2008. find
  4. Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc’Aurelio Ranzato, and Arthur Szlam. Real or Fake? Learning to Discriminate Machine from Human Generated Text. ArXiv, abs/1906.03351, 2019. — [arXiv:1906.03351](https://arxiv.org/abs/1906.03351) find
  5. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ArXiv, abs/2310.05130, 2023.
  6. Daria Beresneva. Computer-Generated Text Detection Using Machine Learning: A Systematic Review. In International Conference on Applications of Natural Language to Data Bases, 2016. find
  7. Rahul Bhagat and Eduard H. Hovy. Squibs: What Is a Paraphrase? Computational Linguistics, 39:463–472, 2013.
  8. Amrita Bhattacharjee, Tharindu Kumarage, Raha Moraffah, and Huan Liu. ConDA: Contrastive Domain Adaptation for AI-generated Text Detection. ArXiv, abs/2309.03992, 2023. URL https://api.semanticscholar.org/CorpusID:261660497. ↗
  9. Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. On the possibilities of ai-generated text detection. arXiv preprint arXiv:2304.04736, 2023. — [arXiv:2304.04736](https://arxiv.org/abs/2304.04736) find
  10. Yutian Chen, Hao Kang, Vivian Zhai, Liang Li, Rita Singh, and Bhiksha Ramakrishnan. GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content. ArXiv, abs/2305.07969, 2023. URL https://api.semanticscholar.org/CorpusID:258686680. — [arXiv:2305.07969](https://arxiv.org/abs/2305.07969) ↗
  11. Yuei-Lin Chiang, Lu-Ping Chang, Wen-Tai Hsieh, and Wen-Chih Chen. Natural Language Watermarking Using Semantic Substitution for Chinese Text. In International Workshop on Digital Watermarking, 2003. find
  12. Gemma Conroy. How ChatGPT and other AI tools could disrupt scientific publishing. Nature, October 2023a. URL https://www.nature.com/articles/d41586-023-03144-w. ↗
  13. Gemma Conroy. Scientific sleuths spot dishonest ChatGPT use in papers. Nature, September 2023b. URL https://www.nature.com/articles/d41586-023-02477-w. ↗
  14. Evan Crothers, Nathalie Japkowicz, and Herna Viktor. Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. arXiv preprint arXiv:2210.07321, 2022. find
  15. Mack Deguerin. AI-generated nonsense is leaking into scientific journals . Popular Science, March 2024. URL https://www.popsci.com/technology/ai-generated-text-scientific-journals/. ↗
  16. Fabrizio Dell’Acqua, Edward McFowland, Ethan R Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R Lakhani. Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and q… find
  17. Yanai Elazar, Akshita Bhagia, Ian Helgi Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Evan Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, et al. What’s In My Big Data? In The Twelfth International Conference on Learning Representations, 2023. — What's In My Big Data?
  18. Holly Else. Abstracts written by ChatGPT fool scientists. Nature, Jan 2023. URL https://www.nature.com/articles/d41586-023-00056-7. ↗
  19. Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi. TweepFake: About detecting deepfake tweets. Plos one, 16(5):e0251415, 2021.
  20. Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three Bricks to Consolidate Watermarks for Large Language Models. 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6, 2023. find
  21. Jacob G Foster, Andrey Rzhetsky, and James A Evans. Tradition and innovation in scientists’ research strategies. American sociological review, 80(5):875–908, 2015. find
  22. Catherine A Gao, Frederick M Howard, Nikolay S Markov, Emma C Dyer, Siddhi Ramesh, Yuan Luo, and Alexander T Pearson. Comparing scientific abstracts generated by Chat-GPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv… find
  23. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. find
  24. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. GLTR: Statistical Detection and Visualization of Generated Text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 111–116, 2019.
  25. Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and A. S. Bedi. Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey. ArXiv, abs/2310.15264, 2023. — [arXiv:2310.15264](https://arxiv.org/abs/2310.15264) find
  26. Sourojit Ghosh and Aylin Caliskan. ’Person’== Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion. arXiv preprint arXiv:2310.19981, 2023. find
  27. Melissa Heikkilä. How to spot AI-generated text. MIT Technology Review, Dec 2022. URL https://www.technologyreview.com/2022/12/19/1065596/how-to-spot-ai-generated-text/. ↗
  28. Xiaobing Hu, Pin-Yu Chen, and Tsung-Yi Ho. RADAR: Robust AI-Text Detection via Adversarial Learning. ArXiv, abs/2307.03838, 2023a. URL https://api.semanticscholar.org/CorpusID:259501842. ↗
  29. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased Watermark for Large Language Models. ArXiv, abs/2310.10669, 2023b. — [arXiv:2310.10669](https://arxiv.org/abs/2310.10669) find
  30. ICML. Clarification on large language model policy LLM. https://icml.cc/Conferences/2023/llm-policy, 2023. ↗
  31. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650, 2019.
  32. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks VS Lakshmanan. Automatic detection of machine generated text: A critical survey. arXiv preprint arXiv:2011.01314, 2020.
  33. Samantha Murphy Kelly. ChatGPT creator pulls AI detection tool due to ‘low rate of accuracy’. CNN Business, Jul 2023. URL https://www.cnn.com/2023/07/25/tech/openai-ai-detection-tool/index.html. ↗
  34. Saurabh Khanna, Jon Ball, Juan Pablo Alperin, and John Willinsky. Recalibrating the scope of scholarly publishing: A modest step in a vast decolonization process. Quantitative Science Studies, 3(4):912–930, 12 2022. ISSN 2641-3337. doi: 10.1162/qss a 00228. URL https://doi.org/10.1162/qss a 00228. — [doi:10.1162/qss](https://doi.org/10.1162/qss) ↗
  35. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. International Conference on Machine Learning, 2023.
  36. Jan Hendrik Kirchner, Lama Ahmad, Scott Aaronson, and Jan Leike. New AI classifier for indicating AI-written text, 2023. URL https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text. ↗
  37. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust Distortion-free Watermarks for Language Models. ArXiv, abs/2307.15593, 2023.
  38. Thomas Lavergne, Tanguy Urvoy, and François Yvon. Detecting Fake Content with Relative Entropy Scoring. Pan, 2008. find
  39. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. Deepfake Text Detection in the Wild. ArXiv, abs/2305.13242, 2023. URL https://api.semanticscholar.org/CorpusID:258832454. — [arXiv:2305.13242](https://arxiv.org/abs/2305.13242) ↗
  40. Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Y. Zou. GPT detectors are biased against non-native English writers. ArXiv, abs/2304.02819, 2023a.
  41. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, et al. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv preprint arXiv:2310.01783, 2023b.
  42. Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland, and James Y. Zou. Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews. arXiv preprint arXi…
  43. Ryan Liu and Nihar B Shah. Reviewergpt? an exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622, 2023.
  44. Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Yu Lan, and Chao Shen. CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning. ArXiv, abs/2212.10341, 2022. URL https://api.semanticscholar.org/CorpusID:254877728. — [arXiv:2212.10341](https://arxiv.org/abs/2212.10341) ↗
  45. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv, abs/1907.11692, 2019.
  46. MacroPolo. The Global AI Talent Tracker, 2024. digital-projects/the-global-ai-talent-tracker/. find
  47. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ArXiv, abs/2301.11305, 2023a.
  48. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. DetectGPT: Zero-shot machine-generated text detection using probability curvature. arXiv preprint arXiv:2301.11305, 2023b.
  49. Paulina Okunytė. Google search exposes academics using ChatGPT in research papers. Cybernews, November 2023. URL https://cybernews.com/news/academic-cheating-chatgpt-openai/. ↗
  50. URL https://macropolo.org/OpenAI. GPT-2: 1.5B release. https://openai.com/research/gpt-2-1-5b-release, 2019. Accessed: 2019-11-05. ↗
  51. Ivan Oransky and Adam Marcus. Papers and peer reviews with evidence of ChatGPT writing . Retraction Watch, 2024. URL https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/. ↗
  52. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020.
  53. Vinu Sankar Sadasivan, Aounon Kumar, S. Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can AI-Generated Text be Reliably Detected? ArXiv, abs/2303.11156, 2023.
  54. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning, pp. 29971–30004. PMLR, 2023.
  55. Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, and Cho-Jui Hsieh. Red Teaming Language Model Detectors with Language Models. ArXiv, abs/2305.19713, 2023.
  56. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493, 2023.
  57. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203, 2019.
  58. H. Holden Thorp. Chatgpt is fun, but not an author. Science, 379(6630):313–313, 2023. doi: 10.1126/science.adg7879. URL https://www.science.org/doi/abs/10.1126/science. adg7879. — [doi:10.1126/science.adg7879](https://doi.org/10.1126/science.adg7879) ↗
  59. Mercan Topkara, Giuseppe Riccardi, Dilek Z. Hakkani-Tür, and Mikhail J. Atallah. Natural language watermarking: challenges in building a practical system. In Electronic imaging, 2006a. find
  60. Umut Topkara, Mercan Topkara, and Mikhail J. Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Workshop on Multimedia & Security, 2006b. find
  61. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  62. Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, S. Barannikov, Irina Piontkovskaya, Sergey I. Nikolenko, and Evgeny Burnaev. Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts. ArXiv, abs/2306.04723, 2023.
  63. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. Authorship Attribution for Neural Text Generation. In Conference on Empirical Methods in Natural Language Processing, 2020.
  64. Dann. Van Rossum. Generative AI Top 150: The World’s Most Used AI Tools. https://www.flexos.work/learn/generative-ai-top-150, February 2024. ↗
  65. Veniamin Veselovsky, Manoel Horta Ribeiro, and Robert West. Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks. arXiv preprint arXiv:2306.07899, 2023.
  66. James Vincent. ‘As an AI language model’: the phrase that shows how AI is pollulating the web. The Verge, Apr 2023. URL https://www.theverge.com/2023/4/25/23697218/ai-generated-spam-fake-user-reviews-as-an-ai-language-model. ↗
  67. Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut, and Lorna Waddington. Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1):26, 2023. ISSN 1833-2595. doi: 10.1007/s40979-… — [doi:10.1007/s40979-023-00146-z](https://doi.org/10.1007/s40979-023-00146-z) find
  68. Max Wolff. Attacking Neural Text Detectors. ArXiv, abs/2002.11768, 2020. — [arXiv:2002.11768](https://arxiv.org/abs/2002.11768) find
  69. Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. DiPmark: A Stealthy, Efficient and Resilient Watermark for Large Language Models. ArXiv, abs/2310.07710, 2023. — [arXiv:2310.07710](https://arxiv.org/abs/2310.07710) find
  70. Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang, and Haifeng Chen. DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. ArXiv, abs/2305.17359, 2023a. — [arXiv:2305.17359](https://arxiv.org/abs/2305.17359) find
  71. Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Ruth Petzold, William Yang Wang, and Wei Cheng. A Survey on Detection of LLMs-Generated Content. ArXiv, abs/2310.15654, 2023b. find
  72. Kiyoon Yoo, Wonhyuk Ahn, Jiho Jang, and No Jun Kwak. Robust Multi-bit Natural Language Watermarking through Invariant Features. In Annual Meeting of the Association for Computational Linguistics, 2023. find
  73. Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Weiming Zhang, and Neng H. Yu. GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance. ArXiv, abs/2305.12519, 2023. URL https://api.semanticscholar.org/CorpusID:258833423. — [arXiv:2305.12519](https://arxiv.org/abs/2305.12519) ↗
  74. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending Against Neural Fake News. ArXiv, abs/1905.12616, 2019.
  75. Yi-Fan Zhang, Zhang Zhang, Liang Wang, Tien-Ping Tan, and Rong Jin. Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors. ArXiv, abs/2312.12918, 2023. — [arXiv:2312.12918](https://arxiv.org/abs/2312.12918) find
  76. Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting Language Generation Models via Invisible Watermarking. In Proceedings of the 40th International Conference on Machine Learning, pp. 42187–42199, 2023. find
  77. Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable Robust Watermarking for AI-Generated Text. In International Conference on Learning Representations (ICLR), 2024a.
  78. Xuandong Zhao, Lei Li, and Yu-Xiang Wang. Permute-and-Flip: An optimally robust and watermarkable decoder for LLMs. arXiv preprint arXiv:2402.05864, 2024b. — [arXiv:2402.05864](https://arxiv.org/abs/2402.05864) find
  79. Estimated Fraction of LLM-Modified Sentences in Introductions find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liang-2024-mapping-increasing-use-llms/