📄 Ground Truths in Suicide Research: The Current State of AI-Based Suicide Detection in Social Media

Summary

AI models that flag suicide risk from social media often report strong results. This paper asks what those models are actually trained to recognize. The authors pooled two sources. The first is an umbrella review by Abdelmoteleb et al. (2025) covering 22 systematic reviews; eight of these were relevant and yielded 69 primary studies. The second is the authors' own ongoing review, preregistered on OSF, which adds 126 studies from 2022 onward. Across the resulting 195 studies, they examined platforms, data types, languages, dataset reuse and labeling practices.

The field is growing fast, but it draws on a narrow base: Reddit, English, text only, and the same few datasets used repeatedly. Most importantly, very few studies confirm suicide risk at the level of the individual. Labels usually come from proxies such as which subreddit a post appeared in, or from expert judgments of what a post says. As a result, models mostly learn to classify suicidal-sounding language rather than to identify people at risk. This means high accuracy scores should be read cautiously. Future progress depends on measures that reflect suicide risk as people actually experience it.

Key points

• The corpus contains 195 studies, about 43% of them (n = 84) from 2024–2026. A supplementary dataset documents each study's characteristics.
• Reddit dominates as a data source:
  • As the sole platform, Reddit accounts for 94 studies (48%), followed by Twitter/X with 46 (24%) and Sina Weibo with 19 (10%).
  • Counting multi-platform studies, Reddit appears in 113 and Twitter in 64.
  • Facebook and Instagram are largely missing.
• 98% of studies (n = 191) rely on text alone, and none systematically use audio, video or behavioral metadata. English-only work makes up 78% (n = 153), and Chinese is the only other notable language (n = 20).
• One public dataset of roughly 232,000 Reddit posts is reused across many studies. It is labeled purely by subreddit membership (e.g., r/SuicideWatch versus general communities).
• Only 14 studies (7.2%) validated risk at the individual level, for example with self-report instruments or the Columbia Suicide Severity Rating Scale. These studies consistently report lower model performance.