Elamur Research⌕
☰
home / papers / 2026 / Detecting Risky Behavior Related to Alcohol and Drug Use within Adolescents' Private Messenger Conversations

Detecting Risky Behavior Related to Alcohol and Drug Use within Adolescents' Private Messenger Conversations

Summary

Most automated detection of substance use relies on medical records or public platforms like X and Reddit. These sources miss how teenagers talk in private. This paper addresses detecting alcohol and drug talk in adolescents' private chats, mostly written in Czech. Twenty-two teenagers aged 13–17 donated a year of Meta Messenger history. The text was anonymized and split into conversations wherever there was a 60-minute pause. Two trained annotators then labelled part of it using a manual refined to capture youth slang, and a social scientist resolved uncertain cases to form a gold standard. Positive cases turned out to be extremely rare. The authors fine-tuned BERT-based classifiers, tested text-preprocessing variants and compared the results with open-source generative LLMs prompted with Czech-specific examples. The best BERT-based model reached an F1 of 0.817. Generative LLMs performed comparably and can be adapted to other languages, including English. An error analysis identified slang and lookalike words as the main pitfalls. The study shows that this social-science task is feasible in a low-resource language and could support parental mediation and parental control tools.

Key points

  • Corpus scale: The corpus has 1,260,492 utterances in 2,807 chat files (90,422 conversations) from 2,165 authors, covering September 2014 to December 2020.
  • Annotation: Annotators labelled 35,000 conversations containing 272,465 utterances. Only 2,301 utterances (0.845%) were tagged positive by at least one annotator. Inter-annotator agreement was Cohen's κ = 0.609.
  • Preprocessing: Combining capitalization handling with removal of redundant tags gave the best result in these experiments (F1 ≈ 0.818). Removing emoji or stop words lowered performance.
  • BERT vs. LLMs: Open-source generative LLMs matched the BERT-based models. Their prompts needed explicit guidance on Czech slang and cultural differences; for example, "420" has no marijuana connotation in Czech.
  • Typical errors:
    • Both model types missed slang terms such as "skéro" and "zkuřka".
    • LLMs produced false positives on similar-looking words, such as "nadrzenej" (horny), "cigos" (a slur) and "mariin" (a form of the name Marie).
    • LLMs also flagged nicknames that merely mention beer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2rjd9g5nbkxv

@inproceedings{plhak2026detecting,
  title = {{Detecting Risky Behavior Related to Alcohol and Drug Use within Adolescents' Private Messenger Conversations}},
  author = {Plhák, Jaromír and Lebedíková, Michaela and Sotolar, Ondrej and Smahel, David},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2rjd9g5nbkxv},
  url = {https://doi.org/10.63317/2rjd9g5nbkxv}
}

References

64 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammad H. Afzali, Matthew Sunderland, Sherry Stewart, Benoit Masse, Jean Seguin, Nicola Newton, Maree Teesson, and Patricia Conrod. 2019. Machine-learning prediction of adolescent alcohol use: a cross-study, cross-cultural validation. Addiction, 114(4):662–671. find
  2. Jon-Patrick Allem, Likhit Dharmapuri, Jennifer B Unger, and Tess Boley Cruz. 2018. Characterizing juul-related posts on twitter. Drug and alcohol dependence, 190:1–5. find
  3. Ashwaq Alsoubai. 2023. A human-centered approach to improving adolescent real-time online risk detection algorithms. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA ’23, New York, NY, USA. Association for Computing Machinery. find
  4. Yin Aphinyanaphongs, Bisakha Ray, Alexander Statnikov, and Paul Krebs. 2014. Text classification for automatic detection of alcohol use-related tweets: A feasibility study. In Proceedings of the 2014 IEEE 15th international conference on information reuse and integration (IEEE IRI 2014), pages 93–97… find
  5. Astrid B. Bräker and Renate Soellner. 2016. Alcohol drinking cultures of European adolescents. European Journal of Public Health, 26(4):581– 586. find
  6. Lorena Charrier, Saskia van Dorsselaer, Natale Canale, Tibor Baska, Biljana Kilibarda, Rosanna Irene Comoretto, Tommaso Galeotti, Judith Brown, and Alessio Vieno. 2021. A focus on adolescent substance use in europe, central asia and canada. Health Behaviour in School-aged Children international repo… find
  7. Annie T Chen, Shu-Hong Zhu, and Mike Conway. 2015. What online communities can tell us about electronic cigarettes and hookah use: a study using text mining and visualization techniques. Journal of medical Internet research, 17(9):e220. find
  8. William J. Clancey. 1979. Transfer of Rule-Based Expertise through a Tutorial Dialogue. Ph.D. diss., Dept. of Computer Science, Stanford Univ., Stanford, Calif. find
  9. William J. Clancey. 1983. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. In Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83), pages 556–560, Menlo Park, Calif. IJCAI Organization. find
  10. William J. Clancey. 1984. Classification Problem Solving. In Proceedings of the Fourth National Conference on Artificial Intelligence, pages 45– 54, Menlo Park, Calif. AAAI Press. find
  11. William J. Clancey. 2021. The Engineering of Qualitative Models. Forthcoming. find
  12. Mike Conway, Mengke Hu, and Wendy W Chapman. 2019. Recent advances in using natural language processing to address public health research questions using social media and con-sumergenerated data. Yearbook of medical informatics, 28(01):208–217. find
  13. Raminta Daniulaityte, Lu Chen, Francois R Lamy, Robert G Carlson, Krishnaprasad Thirunarayan, Amit Sheth, et al. 2016. “when ‘bad’is ‘good’”: identifying personal communication and sentiment in drug-related tweets. JMIR public health and surveillance, 2(2):e6327. find
  14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  15. Steriani Elavsky, Jana Blahošová, Michaela Lebedíková, Michał Tkaczyk, Martin Tancos, Jaromír Plhák, Ondřej Sotolář, David Smahel, et al. 2022. Researching the links between smartphone behavior and adolescent well-being with the future-wp4 (modeling the future: understanding the impact of technology… find
  16. Gervilla García Elena, Jimenez López Rafael, Juan Jose, Montaño Moreno, Sese Abad Albert, Cajal Blasco Berta, and Palmer Pol Alfonso. 2009. The methodology of data mining. an application to alcohol consumption in teenagers. Adicciones, 21(1):65–80. find
  17. Robert Engelmore and Anthony Morgan, editors. 1986. Blackboard Systems. Addison-Wesley, Reading, Mass. find
  18. Rebecca Fiebrink and Marco Gillies. 2018. Introduction to the special issue on human-centered machine learning. find
  19. FORCE11. 2020. The fair data principles. https://force11.org/info/the-fair-data-principles/. ↗
  20. Johann Gagnon-Bartsch and Yotam Shem-Tov. 2019. The classification permutation test. The Annals of Applied Statistics, 13(3):1464–1483. find
  21. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Communications of the ACM, 64(12):86–92. find
  22. Marco Gillies, Rebecca Fiebrink, Atau Tanaka, Jérémie Garcia, Frédéric Bevilacqua, Alexis Heloir, Fabrizio Nunnari, Wendy Mackay, Saleema Amershi, Bongshin Lee, et al. 2016. Human-centred machine learning. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing sys… find
  23. Alison Giovanelli, Jonathan Rowe, Madelynn Taylor, Mark Berna, Kathleen P Tebb, Carlos Pe-nilla, Marianne Pugatch, James Lester, and Elizabeth M Ozer. 2023. Supporting adolescent engagement with artificial intelligence–driven digital health behavior change interventions. Journal of medical Internet… find
  24. Elizabeth M Glowacki, Joseph B Glowacki, and Gary B Wilcox. 2018. A text-mining analysis of the public’s reactions to the opioid crisis. Substance abuse, 39(2):129–133. find
  25. Yuting Guo, Anthony Ovadje, Mohammed Ali Al-Garadi, and Abeed Sarker. 2024. Evaluating large language models for health-related text classification tasks with public social media data. Journal of the American Medical Informatics Association, 31(10):2181–2189. find
  26. Diane Warner Hasling, William J. Clancey, and Glenn Rennels. 1984. Strategic explanations for a diagnostic consultation system. International Journal of Man-Machine Studies, 20(1):3–19. find
  27. Diane Warner Hasling, William J. Clancey, Glenn R. Rennels, and Thomas Test. 1983. Strategic Explanations in Consultation—Duplicate. The International Journal of Man-Machine Studies, 20(1):3–19. find
  28. Tom Huang, Anas Elghafari, Kunal Relia, and Rumi Chunara. 2017. High-resolution temporal representations of alcohol and tobacco behaviors from social media data. Proceedings of the ACM on human-computer interaction, 1(CSCW):1–26. find
  29. Myrna Hurtado, Anna Siefkas, Misty M Attwood, Zohora Iqbal, and Jana Hoffman. 2022. Machine learning applications and advancements in alcohol use disorder: A systematic review. Preprint on webpage at https://www.medrxiv.org/content/early/2022/06/07/2022.06.06.22276057. ↗
  30. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, page 2. Minneapolis, Minnesota. find our card
  31. David Li, Kartik Gupta, Mousumi Bhaduri, Paul Sathiadoss, Sahir Bhatnagar, and Jaron Chong. 2024. Comparative diagnostic accuracy of gpt-4o and llama 3-70b: Proprietary vs. open-source large language models in radiology. Clinical Imaging, page 110382. find
  32. Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. 2024. Data-efficient fine-tuning for llm-based recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 365–374, New York… find
  33. Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364. find
  34. Tim Mackey, Janani Kalyanam, Josh Klugman, Ella Kuzmenko, and Rashmi Gupta. 2018. Solution to detect, classify, and report illicit online marketing and sales of controlled substances via twitter: using machine learning and web forensics to combat digital opioid access. Journal of medical Internet re… find
  35. Tim K Mackey and Janani Kalyanam. 2017. Detection of illicit online sales of fentanyls via twitter. F1000Research, 6:5. find
  36. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276– 282. find
  37. Meredith C Meacham, Michael J Paul, and Danielle E Ramo. 2018. Understanding emerging forms of cannabis use through an online cannabis community: an analysis of relative post volume and subjective highness ratings. Drug and alcohol dependence, 188:364–369. find
  38. MetaLlama. 2024a. Llamaindex. https://www.llamaindex.ai/. ↗
  39. MetaLlama. 2024b. Metallama3. llama.meta.com/llama3/. https:// find
  40. NASA. 2015. Pluto: The ’other’ red planet. https://www.nasa.gov/nh/pluto-the-other-red-planet. Accessed: 2018-12-06. ↗
  41. Jaromír Plhák, Michaela Lebedíková, Michał Tkaczyk, and David Šmahel. 2023. Web-based annotation tool for instant messaging conversations. find
  42. RASLAN 2023 Recent Advances in Slavonic Natural Language Processing, page 3. find
  43. Jaromír Plhák, Ondřej Sotolář, Michaela Lebedíková, and David Šmahel. 2023. Classification of adolescents’ risky behavior in instant messaging conversations. In International Conference on Artificial Intelligence and Statistics, pages 2390–2404, Valencia, Spain. PMLR. find
  44. Jaromír Plhák, Ondřej Sotolář, Michaela Lebedíková, and David Šmahel. 2025. Bert-based model for substance use classifiation. https://huggingface.co/AnnonSubConf. Accessed: 2025-01-14. ↗
  45. Roshan Prakash Rane, Andreas Heinz, and Kerstin Ritter. 2022. AIM in Alcohol and Drug Dependence, pages 1619–1628. Springer International Publishing, Cham. find
  46. Benjamin Joseph Ricard and Saeed Hassanpour. 2021. Deep learning for identification of alcohol-related content on social media (reddit and twitter): exploratory analysis of alcohol-related outcomes. Journal of medical internet research, 23(9):e27314. find
  47. James Rice. 1986. Poligon: A System for Parallel Problem Solving. Technical Report KSL-86-19, Dept. of Computer Science, Stanford Univ. find
  48. Arthur L. Robinson. 1980a. New ways to make microcircuits smaller. Science, 208(4447):1019– 1022. find
  49. Arthur L. Robinson. 1980b. New Ways to Make Microcircuits Smaller—Duplicate Entry. Science, 208:1019–1026. find
  50. Konstantinos I Roumeliotis, Nikolaos D Tselikas, and Dimitrios K Nasiopoulos. 2024. Llms in ecommerce: a comparative analysis of gpt and llama models in product review evaluation. Natural Language Processing Journal, 6:100056. find
  51. V Sanh. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
  52. Fatemeh Shah-Mohammadi and Joseph Finkelstein. 2024. Extraction of substance use information from clinical notes: Generative pretrained transformer–based investigation. JMIR Medical Informatics, 12:e56243. find
  53. Ondřej Sotolář, Jaromír Plhák, and David Šmahel. 2021. Towards personal data anonymization for social messaging. In International Conference on Text, Speech, and Dialogue, pages 281–292, Cham. Springer International Publishing. find
  54. Milan Straka, Jakub Náplava, Jana Straková, and David Samuel. 2021. RobeCzech: Czech RoBERTa, a Monolingual Contextualized Language Representation Model. In Text, Speech, and Dialogue, pages 197–209, Cham. Springer International Publishing. find
  55. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct distillation of lm alignment. find
  56. A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems. find our card
  57. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. find our card
  58. Alejandro L Vázquez, Melanie M Domenech Rodríguez, Tyson S Barrett, Sarah Schwartz, Nancy G Amador Buenabad, Marycarmen N Bustos Gamiño, María de Lourdes Gutiérrez López, and Jorge A Villatoro Velázquez. 2020. Innovative identification of substance use predictors: machine learning in a national samp… find
  59. Maretha Visser and Leigh-Anne Routledge. 2007. Substance abuse and psychological well-being of south african adolescents. South African Journal of Psychology, 37(3):595–615. find
  60. Robert Whelan, Richard Watts, Catherine A Orr, Robert R Althoff, Eric Artiges, Tobias Banaschewski, Gareth J Barker, Arun LW Bokde, Christian Büchel, Fabiana M Carvalho, et al. 2014. Neuropsychosocial profiles of current and future adolescent alcohol misusers. Nature, 512(7513):185–189. find
  61. Zhilin Yang. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237. find our card
  62. Yongcheng Zhan, Ruoran Liu, Qiudan Li, Scott James Leischow, and Daniel Dajun Zeng. 2017. Identifying topics for e-cigarette user-generated contents: a case study from multiple social media platforms. Journal of medical Internet research, 19(1):e24. find
  63. Xinyu Zhang, Jianfeng Zhu, Deric R Kenne, and Ruoming Jin. 2025. Teenager substance use on reddit: Mixed methods computational analysis of frames and emotions. Journal of Medical Internet Research, 27:e59338. find
  64. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/plhak-2026-detecting-risky-behavior-related-alcohol/