Elamur Research⌕
☰
home / papers / 2026 / Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants

Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants

Paper · 2026 · Jaspreet Ranjit, Hyundong Justin Cho, Claire J. Smerdon et al.

Summary

This work examines whether language models can lighten the load on people who code the U.S. National Violent Death Reporting System (NVDRS). NVDRS covers more than 270K suicides from 2003–2019, with free-text CME and LE narratives and over 600 structured variables. Because the records are sensitive, the authors run open-weight models locally.

They first turn official NVDRS codebooks into standardized zero-shot Chain-of-Thought prompts and compare model labels with abstractor labels on 50 binary variables. Llama-3-70B agrees about 85% of the time. An expert review of disagreements on six low-agreement variables found that 38% came from conflicts between the narrative and the structured record, not from model mistakes.

They then propose a human-in-the-loop procedure for building codebooks for new variables, where experts correct only wrong predictions. Applied to victims' contact with legal professionals, it shortens codebook creation from weeks to hours with quality close to a manual codebook. It also shows such contact is common, which points to a nonclinical intervention opportunity missing from the structured fields.

Key points

  • Agreement with abstractors. On D_balanced (500 class-balanced narratives per variable, 25,000 total), mean agreement was:

    • Llama-3-70B: 0.85 (95% CI 0.82–0.87)
    • Qwen2.5-14B: 0.79
    • Qwen2.5-7B and Mistral-7B: 0.73 each
    • Llama-3-8B: 0.71

    A random 1,000-narrative set (D_random) was also used to reflect real-world class skew.

  • Variables evaluated. The 50 variables comprise 36 circumstance and 14 crisis variables, each present in at least 300 cases. The authors note that many of the 600+ NVDRS variables depend on sources they could not access, such as death certificates and toxicology reports.

  • Codebooks from scratch. For 12 existing variables, codebooks generated by the algorithm from only the variable name reached 80% agreement. Prompts built from the official NVDRS codebooks reached 75%.

  • Legal-interaction case study.

    • Validation used 634 expert-annotated narratives, labeled by one expert and two trained annotators over about three months, with inter-annotator agreement of 0.88.
    • The generated and manual codebooks scored nearly the same (F1 of roughly 0.78–0.79).
    • The prevalence figures in the extracted text are inconsistent and should be checked against the original: 10.4% of narratives in the Figure 1 caption, 12.7% in the introduction, and about 96K narratives in the abstract.

  • Privacy and code. No data was sent to API providers, and the code is released at dill-lab.github.io/interventions_lm_assistants/.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Southern CaliforniaLos Angeles · United States175 works · 37,047 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksPsychology · 1,649 worksSocial Psychology · 330 worksClinical Psychology · 21 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Jaspreet Ranjit, Hyundong Justin Cho, Claire J. Smerdon, Yoonsoo Nam, Myles Phung, Jonathan May, John R. Blosnich and Swabha Swayamdipta. 2026. Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.688

@inproceedings{ranjit2026uncovering,
  title = {{Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants}},
  author = {Jaspreet Ranjit, Hyundong Justin Cho, Claire J. Smerdon, Yoonsoo Nam, Myles Phung, Jonathan May, John R. Blosnich and Swabha Swayamdipta},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.688},
  url = {https://doi.org/10.18653/v1/2026.acl-long.688}
}

References

60 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alina Arseniev-Koehler, Susan D Cochran, Vickie M Mays, Kai-Wei Chang, and Jacob G Foster. 2022. Integrating topic modeling and word embedding to characterize violent deaths. Proceedings of the National Academy of Sciences, 119(10):e2108801119. find
  2. Alina Arseniev-Koehler, Jacob Gates Foster, Vickie M Mays, Kai-Wei Chang, and Susan D Cochran. 2021. Aggression, escalation, and other latent themes in legal intervention deaths of non-hispanic black and white men: Results from the 2003–2017 national violent death reporting system. American journal… find
  3. Alina Arseniev-Koehler, Vickie M Mays, Jacob G Foster, Kai-Wei Chang, and Susan D Cochran. 2023. find
  4. Gendered patterns in manifest and latent mental health indicators among suicide decedents: 2003– 2020 national violent death reporting system (nvdrs). American journal of public health, 114(S3):S268– S277. find
  5. Amanda Barany, Nidhi Nasiar, Chelsea Porter, Andres Felipe Zambrano, Alexandra L Andres, Dara Bright, Mamta Shah, Xiner Liu, Sabrina Gao, Jiayi Zhang, and 1 others. 2024. Chatgpt for education research: exploring the potential of large language models for qualitative codebook development. In Interna… find
  6. Janet M Blair, Katherine A Fowler, Shane PD Jack, and Alexander E Crosby. 2016. The national violent death reporting system: overview and future directions. Injury prevention, 22(Suppl 1):i6–i11. find
  7. John Blosnich, Jeanne Ward, Alexandra Haydinger, Melissa Perkins, and Susa De Luca. 2024. Industries of disruption: New avenues for upstream suicide prevention. In APHA 2024 Annual Meeting and Expo. APHA. find
  8. Alison L Cammack. 2024. Vital signs: Suicide rates and selected county-level factors—united states, 2022. MMWR. Morbidity and Mortality Weekly Report, 73. find
  9. Christina Chance, Alina Arseniev-Koehler, Vickie M Mays, Kai-Wei Chang, and Susan D Cochran. 2025. Measuring narrative complexity among suicide deaths in the national violent death reporting system (2003–2021 nvdrs). Information, 16(11):989. find
  10. Bernardo Consoli, Xizhi Wu, Song Wang, Xinyu Zhao, Yanshan Wang, Justin Rousseau, Tom Hartvigsen, Li Shen, Huanmei Wu, Yifan Peng, and 1 others. 2024. Sdoh-gpt: Using large language models to extract social determinants of health (sdoh). arXiv preprint arXiv:2407.17126. — [arXiv:2407.17126](https://arxiv.org/abs/2407.17126) find
  11. Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku. 2023. Llm-in-the-loop: Leveraging large language model for thematic analysis. arXiv preprint arXiv:2310.15100. find
  12. Linh N Dang, Eskira T Kahsay, LaTeesa N James, Lily J Johns, Isabella E Rios, and Briana Mezuk. 2023. Research utility and limitations of textual data in the national violent death reporting system: a scoping review and recommendations. Injury epidemiology, 10(1):23. find
  13. Nabarun Dasgupta. 2021. Ghost in the machine: The emotional gravity of conducting mortality research. find
  14. Judy E Davidson, Gordon Ye, Felicia Deskins, Heather Rizzo, Christine Moutier, and Sidney Zisook. 2021. Exploring nurse suicide by firearms: A mixed-method longitudinal (2003–2017) analysis of death investigations. In Nursing forum, volume 56, pages 264–272. Wiley Online Library. find
  15. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library.
  16. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. Chapman and Hall/CRC. find
  17. Houriyeh Ehtemam, Shabnam Sadeghi Esfahlani, Alireza Sanaei, Mohammad Mehdi Ghaemi, Sadrieh Hajesmaeel-Gohari, Rohaneh Rahimisadegh, Kam-biz Bahaadinbeigy, Fahimeh Ghasemian, and Hassan Shirvani. 2024. Role of machine learning algorithms in suicide risk prediction: a systematic review-meta analysis… find
  18. Ben Fincham, Jonathan Scourfield, and Susanne Langer. 2008. The impact of working with disturbing secondary data: Reading suicide files in a coroner’s office. Qualitative Health Research, 18(6):853–862. find
  19. Barney G. Glaser and Anselm L. Strauss. 1967. Discovery of Grounded Theory Strategies for Qualitative Research. AldineTransaction, London. find
  20. Marco Guevara, Shan Chen, Spencer Thomas, Tafadzwa L Chaunzwa, Idalid Franco, Benjamin H Kann, Shalini Moningi, Jack M Qian, Madeleine Goldstein, Susan Harper, and 1 others. 2024. Large language models to identify social determinants of health in electronic health records. NPJ digital medicine, 7(1)… find
  21. Shivanshu Gupta, Matt Gardner, and Sameer Singh. 2023. Coverage-based example selection for in-context learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13924–13950, Singapore. Association for Computational Linguistics. find
  22. Andrew Halterman and Katherine A Keith. 2024. Codebook llms: Adapting political science codebooks for llm use and adapting llms to follow codebooks. ArXiv. find
  23. Lily Johns, Chuwen Zhong, and Briana Mezuk. 2023. Understanding suicide over the life course using data science tools within a triangulation framework. Journal of psychiatry and brain science, 8(1):e230003. find
  24. Julie M Kafka, Mike D Fliss, Pamela J Trangenstein, Luz McNaughton Reyes, Brian W Pence, and Kathryn E Moracco. 2023. Detecting intimate partner violence circumstance for suicide: development and validation of a tool using natural language processing and supervised machine learning in the national v… find
  25. Julie M Kafka, Kathryn Elizabeth Moracco, Brian W Pence, Pamela J Trangenstein, Mike Dolan Fliss, and Luz McNaughton Reyes. 2024. Intimate partner violence and suicide mortality: a cross-sectional study using machine learning and natural language processing of suicide data from 43 states. Injury Pre… find
  26. Andrew Katz, Gabriella Coloyan Fleming, and Joyce Main. 2024. Thematic analysis with open-source generative ai and machine learning: A new method for inductive qualitative codebook development. arXiv preprint arXiv:2410.03721. — [arXiv:2410.03721](https://arxiv.org/abs/2410.03721) find
  27. Olivia J Kirtley, Kasper van Mens, Mark Hoogendoorn, Navneet Kapur, and Derek De Beurs. 2022. Translating promise into practice: a review of machine learning in suicide research and prevention. The Lancet Psychiatry, 9(3):243–252. find
  28. Klaus Krippendorff. 1970. Estimating the reliability, systematic error and random error of interval data. Educational and psychological measurement, 30(1):61–70. find
  29. Christa D Labouliere, Prabu Vasan, Anni Kramer, Greg Brown, Kelly Green, Mahfuza Rahman, Jamie Kammer, Molly Finnerty, and Barbara Stanley. 2018. “zero suicide”–a model for reducing suicide in united states behavioral healthcare. Suicidologi, 23(1):22. find
  30. Alban Lejeune, Aziliz Le Glaz, Pierre-Antoine Perron, Johan Sebti, Enrique Baca-Garcia, Michel Walter, Christophe Lemey, and Sofian Berrouiguet. 2022. Artificial intelligence and suicide prevention: a systematic review. European psychiatry, 65(1):e19. find
  31. Grace S Liu. 2023. Surveillance for violent deaths—national violent death reporting system, 48 states, the district of columbia, and puerto rico, 2020. MMWR. Surveillance Summaries, 72. find
  32. Kevin Lybarger, Oliver J Bear Don’t Walk IV, Meliha Yetisgen, and Özlem Uzuner. 2023. Advancements in extracting social determinants of health information from narrative text. find
  33. Ruth Melia, Katherine Musacchio Schafer, Megan L Rogers, Emma Wilson-Lemoine, and Thomas Ellis Joiner. 2025. The application of ai to ecological momentary assessment data in suicide research: Systematic review. Journal of Medical Internet Research, 27:e63192. find
  34. Vivek Murthy. 2024. National strategy for suicide prevention. find
  35. Oybek Nazarov, Joseph Guan, Stanford Chihuri, and Guohua Li. 2019. Research utility of the national violent death reporting system: a scoping review. Injury epidemiology, 6:1–12. find
  36. Susan T Parker. 2025b. Supervised natural language processing classification of violent death narratives: Development and assessment of a compact large language model. JMIR AI, 4:e68212. find
  37. Leonard J Paulozzi, J Mercy, Lorraine Frazier, and J Lee Annest. 2004. Cdc’s national violent death reporting system: background and methodology. Injury prevention, 10(1):47–52. find
  38. Mike Perkins and Jasper Roe. 2024. The use of generative ai in qualitative analysis: Inductive thematic analysis with chatgpt. Journal of Applied Learning and Teaching, 7(1). find
  39. John P Pestian, Pawel Matykiewicz, Michelle Linn-Gust, Brett South, Ozlem Uzuner, Jan Wiebe, K Bretonnel Cohen, John Hurdle, and Christopher Brew. 2012. Sentiment analysis of suicide notes: A shared task. Biomedical informatics insights, 5:BII–S9042. find
  40. Alexandra Ralevski, Nadaa Taiyab, Michael Nossal, Lindsay Mico, Samantha N Piekos, and Jennifer Had-lock. 2024. Using large language models to annotate complex cases of social determinants of health in longitudinal clinical records. medRxiv. find
  41. Jaspreet Ranjit, Brihi Joshi, Rebecca Dorn, Laura Petry, Olga Koumoundouros, Jayne Bottarini, Peichen Liu, Eric Rice, and Swabha Swayamdipta. 2024. OATH-frames: Characterizing online attitudes towards homelessness with LLM assistants. In Proceedings of the 2024 Conference on Empirical Methods in Nat… find
  42. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  43. Ian RH Rockett, Eric D Caine, Hilary S Connery, Gail D’Onofrio, David J Gunnell, Ted R Miller, Kurt B Nolte, Mark S Kaplan, Nestor D Kapusta, Christa L Lilly, and 1 others. 2018. Discerning suicide in drug intoxication deaths: Paucity and primacy of suicide notes and psychiatric history. PLoS one, 1… find
  44. Christopher Michael Rytting, Taylor Sorensen, Lisa Argyle, and 1 others. 2023. Towards coding social science datasets with language models. ArXiv:2306.02177. — [arXiv:2306.02177](https://arxiv.org/abs/2306.02177) find
  45. Nicholas Pangakis, Samuel Wolken, and Neil Fasching. 2023. Automated annotation with generative ai requires validation. arXiv preprint arXiv:2306.00176.
  46. Mark Sinyor, Morton Silverman, Jane Pirkis, and Keith Hawton. 2024. The effect of economic downturn, financial hardship, unemployment, and relevant government responses on suicide. The Lancet Public Health, 9(10):e802–e806. find
  47. Susan T Parker. 2025a. Assessing supervised natural language processing (nlp) classification of violent death narratives: Development and assessment of a compact large language model (llm) approach. medRxiv, pages 2025–01. find
  48. Malinda Steenkamp, Lorraine Frazier, N Lipskiy, M De-Berry, S Thomas, L Barker, and Debra Karch. 2006. The national violent death reporting system: an exciting new tool for public health surveillance. Injury prevention, 12(suppl 2):ii3–ii5. find
  49. Robert H Tai, Lillian R Bentley, Xin Xia, Jason M Sitt, Sarah C Fankhauser, Ana M Chicas-Mosier, and Barnas G Monteith. 2024. An examination of the use of large language models to aid analysis of textual data. International Journal of Qualitative Methods, 23:16094069241231168. find
  50. Steven M Teutsch and R Elliott Churchill. 2000. Principles and practice of public health surveillance. Oxford University Press. find
  51. Weipeng Zhou, Laura C Prater, Evan V Goldstein, Stephen J Mooney, and 1 others. 2023. Identifying rare circumstances preceding female firearm suicides: validating a large language model approach. JMIR mental health, 10(1):e49359. find
  52. Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2024. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291.
  53. Angelina Wang, Jamie Morgenstern, and John P Dickerson. 2024a. Large language models that replace human participants can harmfully misportray and flatten identity groups. arXiv preprint arXiv:2402.01908. — [arXiv:2402.01908](https://arxiv.org/abs/2402.01908) find
  54. Song Wang, Yifang Dang, Zhaoyi Sun, Ying Ding, Jyotishman Pathak, Cui Tao, Yunyu Xiao, and Yifan Peng. 2023. An nlp approach to identify sdoh-related circumstance and suicide crisis from death investigation narratives. Journal of the American Medical Informatics Association, 30(8):1408–1417. find
  55. Song Wang, Yishu Wei, Haotian Ma, Max Lovitt, Kelly Deng, Yuan Meng, Zihan Xu, Jingze Zhang, Yunyu Xiao, Ying Ding, and 1 others. 2025. A multi-stage large language model framework for extracting suicide-related social determinants of health. Communications Medicine, 5(1):404. find
  56. Song Wang, Yiliang Zhou, Ziqiang Han, Cui Tao, Yunyu Xiao, Ying Ding, Joydeep Ghosh, and Yifan Peng. 2024b. A natural language processing approach to detect inconsistencies in death investigation notes attributing suicide circumstances. Communications Medicine, 4(1):199. find
  57. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  58. Rebecca F Wilson. 2022. Surveillance for violent deaths—national violent death reporting system, 42 states, the district of columbia, and puerto rico, 2019. MMWR. Surveillance Summaries, 71. find
  59. Ziang Xiao, Xingdi Yuan, Q Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023. Supporting qualitative analysis with large language models: Combining codebook with gpt-3 for deductive coding. In Companion proceedings of the 28th international conference on intelligent user interfaces, pages 7… find
  60. Richard Li Xu, Song Wang, Zewei Wang, Yuhan Zhang, Yunyu Xiao, Jyotishman Pathak, David Hodge, Yan Leng, S Craig Watkins, Ying Ding, and 1 others. 2024. Analyzing social factors to enhance suicide prevention across population groups. In 2024 IEEE 12th International Conference on Healthcare Informati… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ranjit-2026-uncovering-intervention-opportunities-suicide-prevention/