Elamur Research⌕
☰
home / papers / 2023 / Detecting Suicidality in Arabic Tweets Using Machine Learning and Deep Learning Techniques

Detecting Suicidality in Arabic Tweets Using Machine Learning and Deep Learning Techniques

Summary

Automatic screening of social media for suicidal ideation has mostly been studied for English and Chinese. Arabic had only one earlier attempt, and it relied on translated English data. The authors address this by building what they describe as the first native Arabic suicidality corpus from Twitter: 5,719 tweets written in several dialects from multiple Arabic-speaking countries, labelled by two annotators, one of them a psychology PhD. They then run a broad comparison. Five classical classifiers (Naïve Bayes, SVM, KNN, Random Forest, XGBoost) are each combined with six text representations, ranging from bag of words and word- or character-level TF-IDF to Arabic word2vec and FastText embeddings. Three pre-trained transformers (AraBert, AraELECTRA, AraGPT2) are fine-tuned on the same binary task. Character n-gram TF-IDF is the strongest representation for every classical model, and SVM and RF reach 86% accuracy with it. All three transformers still do better, and AraBert leads with 91% accuracy and 88% F1. The work gives a dataset and a benchmark for mental-health monitoring in a morphologically rich, dialect-heavy language.

Key points

  • Dataset: 5,719 Arabic tweets collected from 23 August 2021 to 21 April 2022, with 1,429 labelled Suicidal and 4,290 Non-Suicidal (about a quarter positive).
  • Features tested with the classical models: BOW, unigram TF-IDF, n-gram TF-IDF, character n-gram TF-IDF, Arabic word2vec and FastText. XGBoost was run with 200 and 300 estimators.
  • Best classical result: SVM and RF with character n-gram TF-IDF, both at 86% accuracy. The abstract reports 79% F1 for this result.
  • Transformers were fine-tuned with batch size 16, 5 epochs, learning rate 2e-5 and adam_epsilon 1e-8:
    • AraBert: 88% precision, 89% recall, 88% F1, 91% accuracy.
    • AraGPT2: 86% F1, 89% accuracy.
    • AraELECTRA: 85% F1, 88% accuracy.
  • Evaluation used precision, recall, F1 and accuracy, plus ROC curves for the classical models and a confusion matrix for AraBert.

Interesting ideas

  • I1. Sub-word character n-grams beat word-level and embedding features for every classical classifier, which suggests they cope well with heavy morphology and dialect spelling variation. — For Mova (tokens): Character n-gram features could serve as a fallback signal in the tagger or lemmatizer for unknown words, misspellings and informal social-media English.
  • I2. The authors cross every feature type with every classifier in one grid, so they can tell whether a gain comes from the representation or from the model. — For Mova (learn): Mova's evaluations could likewise cross component variants (e.g. parser × frame extractor) to show which stage actually drives an improvement.
  • I3. Accuracy and F1 diverge noticeably on this skewed dataset, so reporting both reveals how a model handles the rare positive class. — For Mova (learn): Mova's evaluation reports should pair accuracy with per-class F1 on rare phenomena (irony, negated events) so majority-class behaviour cannot hide failures.
  • I4. Labels for a sensitive psychological category come partly from a domain expert rather than only from general annotators. — For Mova (frames): Gold data for subtle pragmatic labels (speech acts, implicature, irony) could include an expert-reviewed slice as an anchor against which silver or LLM-teacher labels are checked.
  • I5. Several pre-trained models are fine-tuned with one identical hyperparameter setup, which makes the comparison between them fair and cheap to reproduce. — For Mova (tree): When comparing gparse encoder backbones, a single fixed fine-tuning recipe would isolate the effect of the backbone itself on LAS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.00246 · arXiv 2309.00246

@misc{abdulsalam2023detecting,
  title = {{Detecting Suicidality in Arabic Tweets Using Machine Learning and Deep Learning Techniques}},
  author = {Abdulsalam, Asma and Alhothali, Areej and Al-Ghamdi, Saleh},
  year = {2023},
  doi = {10.48550/arxiv.2309.00246},
  eprint = {2309.00246},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.00246}
}

References

62 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Astoveza, G., Obias, R.J.P., Palcon, R.J.L., Rodriguez, R.L., Fabito, B.S., Octaviano, M.V.: Suicidal behavior detection on twitter using neural network. In: TENCON 2018 - 2018 IEEE Region 10 Conference, pp. 0657–0662 (2018). https://doi.org/10.1109/TENCON.2018.8650162 ↗
  2. De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., Kumar, M.: Discovering shifts to suicidal ideation from mental health content in social media. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI ’16, pp. 2098–2110. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2858036.2858207 . https://doi.org/10.1145/2858036.2858207 ↗
  3. Nock, M.K., Borges, G., Bromet, E.J., Cha, C.B., Kessler, R.C., Lee, S.: Suicide and suicidal behavior. Epidemiologic reviews 30(1), 133–154 (2008) find
  4. Beck, A.T., Kovacs, M., Weissman, A.: Assessment of suicidal intention: the scale for suicide ideation. Journal of consulting and clinical psychology 47(2), 343 (1979) find
  5. Liu, D., Fu, Q., Wan, C., Liu, X., Jiang, T., Liao, G., Qiu, X., Liu, R.: Suicidal ideation cause extraction from social texts. IEEE Access 8, 169333–169351 (2020) find
  6. Weber, A.N., Michail, M., Thompson, A., Fiedorowicz, J.G.: Psychiatric emergencies: assessing and managing suicidal ideation. Medical Clinics 101(3), 553–571 (2017) find
  7. Huang, X., Zhang, L., Chiu, D., Liu, T., Li, X., Zhu, T.: Detecting suicidal ideation in chinese microblogs with psychological lexicons. In: 2014 IEEE 11th Intl Conf on Ubiquitous Intelligence and Computing and 2014 IEEE 11th Intl Conf on Autonomic and Trusted Computing and 2014 IEEE 14th Intl Conf on Scalable Computing and Communications and Its Associated Workshops, pp. 844–849 (2014). IEEE find
  8. Rajesh Kumar, E., Rama Rao, K., Nayak, S.R., Chandra, R.: Suicidal ideation prediction in twitter data using machine learning techniques. Journal of Interdisciplinary Mathematics 23(1), 117–125 (2020) find
  9. Ramírez-Cifuentes, D., Freire, A., Baeza-Yates, R., Puntí, J., Medina-Bravo, P., Velazquez, D.A., Gonfaus, J.M., Gonzàlez, J.: Detection of suicidal ideation on social media: multimodal, relational, and behavioral analysis. Journal of medical internet research 22(7), 17758 (2020) find
  10. O’Dea, B., Wan, S., Batterham, P.J., Calear, A.L., Paris, C., Christensen, H.: Detecting suicidality on twitter. Internet Interventions 2(2), 183–188 (2015) https://doi.org/10.1016/j.invent.2015.03.005 ↗
  11. Burnap, P., Colombo, W., Scourfield, J.: Machine classification and analysis of suicide-related communication on twitter. In: Proceedings of the 26th ACM Conference on Hypertext and Social Media. HT ’15, pp. 75–84. Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2700171.2791023 . https://doi.org/10.1145/2700171.2791023 ↗
  12. Vioules, M.J., Moulahi, B., Azé, J., Bringay, S.: Detection of suicide-related posts in twitter data streams. IBM Journal of Research and Development 62(1), 7–1 (2018) find
  13. Moulahi, B., Azé, J., Bringay, S.: Dare to care: a context-aware framework to track suicidal ideation on social media. In: International Conference on Web Information Systems Engineering, pp. 346–353 (2017). Springer find
  14. Ma, Y., Cao, Y.: Dual attention based suicide risk detection on social media. In: 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp. 637–640 (2020). IEEE find
  15. Narynov, S., Mukhtarkhanuly, D., Kerimov, I., Omarov, B.: Comparative analysis of supervised and unsupervised learning algorithms for online user content suicidal ideation detection. Journal of Theoretical and Applied Information Technology 97(22), 3304–3317 (2019) find
  16. Du, J., Zhang, Y., Luo, J., Jia, Y., Wei, Q., Tao, C., Xu, H.: Extracting psychiatric stressors for suicide from social media using deep learning. BMC medical informatics and decision making 18(2), 77–87 (2018) find
  17. Tadesse, M.M., Lin, H., Xu, B., Yang, L.: Detection of suicide ideation in social media forums using deep learning. Algorithms 13(1) (2020) https://doi.org/10.3390/a13010007 ↗
  18. Fahey, R.A., Boo, J., Ueda, M.: Covariance in diurnal patterns of suicide-related expressions on twitter and recorded suicide deaths. Social Science & Medicine 253, 112960 (2020) https://doi.org/10.1016/j.socscimed.2020.112960 ↗
  19. Ji, S., Yu, C.P., Fung, S.-f., Pan, S., Long, G.: Supervised learning for suicidal ideation detection in online user content. Complexity 2018 (2018) find
  20. Chiroma, F., Liu, H., Cocea, M.: Suiciderelated text classification with prism algorithm. In: 2018 International Conference on Machine Learning and Cybernetics (ICMLC), vol. 2, pp. 575–580 (2018). IEEE find
  21. Benlaaraj, O., El Jaafari, I., Ellahyani, A., Boutaayamou, I.: Prediction of suicidal ideation in a new arabic annotated dataset. In: 2022 9th International Conference on Wireless Networks and Mobile Communications (WINCOM), pp. 1–5 (2022). https://doi.org/10.1109/WINCOM55661.2022.9966481 ↗
  22. Alqurashi, S., Alhindi, A., Alanazi, E.: Large Arabic Twitter Dataset on COVID-19. arXiv (2020). https://doi.org/10.48550/ARXIV.2004.04315 . https://arxiv.org/abs/2004.04315 ↗ our card
  23. Boudad, N., Faizi, R., Oulad Haj Thami, R., Chiheb, R.: Sentiment analysis in arabic: A review of the literature. Ain Shams Engineering Journal 9(4), 2479–2490 (2018) https://doi.org/10.1016/j.asej.2017.04.007 ↗
  24. Farghaly, A., Shaalan, K.: Arabic natural language processing: Challenges and solutions. ACM Transactions on Asian Language Information Processing 8(4) (2009) https://doi.org/10.1145/1644879.1644881 ↗
  25. Elnagar, A.: Investigation on sentiment analysis for arabic reviews. In: 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), pp. 1–7 (2016). https://doi.org/10.1109/AICCSA.2016.7945623 ↗
  26. Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: Liwc and computerized text analysis methods. Journal of Language and Social Psychology 29(1), 24–54 (2010) https://doi.org/10.1177/0261927X09351676 https://doi.org/10.1177/0261927X09351676 ↗
  27. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26 (2013) find
  28. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014) find
  29. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. Journal of machine Learning research 3(Jan), 993–1022 (2003) find our card
  30. CHIROMA, F., LIU, H., COCEA, M.: Text classification for suicide related tweets. In: 2018 International Conference on Machine Learning and Cybernetics (ICMLC), vol. 2, pp. 587–592 (2018). https://doi.org/10.1109/ICMLC.2018.8527039 ↗
  31. Huang, Y., Liu, X., Zhu, T.: Suicidal ideation detection via social media analytics. In: Milošević, D., Tang, Y., Zu, Q. (eds.) Human Centered Computing, pp. 166–174. Springer, Cham (2019) find
  32. Sakib, T.H., Ishak, M., Jhumu, F.F., Ali, M.A.: Analysis of suicidal tweets from twitter using ensemble machine learning methods. In: 2021 International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI), pp. 1–7 (2021). https://doi.org/10.1109/ACMI53878.2021.9528252 ↗
  33. Chadha, A., Kaushik, B.: Machine learning based dataset for finding suicidal ideation on twitter. In: 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), pp. 823–828 (2021). https://doi.org/10.1109/ICICV50876.2021.9388638 ↗
  34. Colombo, G.B., Burnap, P., Hodorog, A., Scourfield, J.: Analysing the connectivity and communication of suicidal users on twitter. Computer Communications 73, 291–300 (2016) https://doi.org/10.1016/j.comcom.2015.07.018 . Online Social Networks ↗
  35. Rezig, A.A.: A novel optimizer technique for suicide prediction in twitter environment. In: 2021 International Conference on Information Systems and Advanced Technologies (ICISAT), pp. 1–5 (2021). https://doi.org/10.1109/ICISAT54145.2021.9678419 ↗
  36. Metzler, H., Baginski, H., Niederkrotenthaler, T., Garcia, D.: Detecting potentially harmful and protective suicide-related content on twitter: Machine learning approach. J Med Internet Res 24(8), 34705 (2022) https://doi.org/10.2196/34705 ↗
  37. Haque, R., Islam, N., Islam, M., Ahsan, M.M.: A comparative analysis on suicidal ideation detection using nlp, machine, and deep learning. Technologies 10(3) (2022) https://doi.org/10.3390/technologies10030057 ↗
  38. Chatterjee, M., Samanta, P., Kumar, P., Sarkar, D.: Suicide ideation detection using multiple feature analysis from twitter data. In: 2022 IEEE Delhi Section Conference (DELCON), pp. 1–6 (2022). https://doi.org/10.1109/DELCON54057.2022.9753295 ↗
  39. Chadha, A., Kaushik, B.: Performance Evaluation of Learning Models for Identification of Suicidal Thoughts. The Computer Journal 65(1), 139–154 (2021) https://doi.org/10.1093/comjnl/bxab060 https://academic.oup.com/comjnl/article-pdf/65/1/139/42152304/bxab060.pdf ↗
  40. Mashaly, M.: Suicide notes (2020). https://www.kaggle.com/mohanedmashaly/suicide-notes ↗
  41. Sonu, I.: Suicidal phrases (2020). https://www.kaggle.com/imeshsonu/suicideal-phrases ↗
  42. Valeriano, K., Condori-Larico, A., Sulla-Torres, J.: Detection of suicidal intent in spanish language social networks using machine learning. International Journal of Advanced Computer Science and Applications 11(4) (2020) find
  43. Sidney, S.: Nonparametric statistics for the behavioral sciences. The Journal of Nervous and Mental Disease 125(3), 497 (1957) find
  44. Deepa, D., Tamilarasi, A., et al.: Sentiment analysis using feature extraction and dictionary-based approaches. In: 2019 Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC), pp. 786–790 (2019). IEEE find
  45. Alsafari, S., Sadaoui, S., Mouhoub, M.: Hate and offensive speech detection on arabic social media. Online Social Networks and Media 19, 100096 (2020) find
  46. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015) find our card
  47. Xu, Y., Liu, J., Yang, W., Huang, L.: Incorporating latent meanings of morphological compositions to enhance word embeddings. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1232– 1242 (2018) find
  48. Li, Q., Shah, S., Liu, X., Nourbakhsh, A.: Data sets: Word embeddings learned from tweets and general data. Proceedings of the International AAAI Conference on Web and Social Media 11(1), 428–436 (2017) https://doi.org/10.1609/icwsm.v11i1.14859 ↗
  49. Naili, M., Chaibi, A.H., Ghezala, H.H.B.: Comparative study of word embedding methods in topic segmentation. Procedia computer science 112, 340–349 (2017) find
  50. Kaibi, I., Satori, H., et al.: A comparative evaluation of word embeddings techniques for twitter sentiment analysis. In: 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems (WITS), pp. 1–4 (2019). IEEE find
  51. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013) ↗
  52. Soliman, A.B., Eissa, K., El-Beltagy, S.R.: Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science 117, 256–265 (2017) https://doi.org/10.1016/j.procs.2017.10.117 . Arabic Computational Linguistics ↗
  53. Al-Rfou, R., Kulkarni, V., Perozzi, B., Skiena, S.: P<sc>OLYGLOT</sc>-NER: Massive Multilingual Named Entity Recognition, pp. 586–594. https://doi.org/10.1137/1.9781611974010.66 . https://epubs.siam.org/doi/abs/10.1137/1.9781611974010.66 ↗
  54. Heikal, M., Torki, M., El-Makky, N.: Sentiment analysis of arabic tweets using deep learning. Procedia Computer Science 142, 114–122 (2018) https://doi.org/10.1016/j. procs.2018.10.466 . Arabic Computational Linguistics ↗
  55. Athiwaratkun, B., Wilson, A.G., Anandkumar, A.: Probabilistic fasttext for multi-sense word embeddings. arXiv preprint arXiv:1806.02901 (2018) ↗
  56. Elhassan, N., Varone, G., Ahmed, R., Gogate, M., Dashtipour, K., Almoamari, H., El-Affendi, M.A., Al-Tamimi, B.N., Albalwy, F., Hussain, A.: Arabic sentiment analysis based on word embeddings and deep learning. Computers 12(6) (2023) https://doi.org/10.3390/computers12060126 ↗
  57. John, G.H., Langley, P.: Estimating continuous distributions in bayesian classifiers. arXiv preprint arXiv:1302.4964 (2013) ↗
  58. Elgeldawi, E., Sayed, A., Galal, A.R., Zaki, A.M.: Hyperparameter tuning for machine learning algorithms used for arabic sentiment analysis. Informatics 8(4) (2021) https://doi.org/10.3390/informatics8040079 ↗
  59. Amoudi, G., Albalawi, R., Baothman, F., Jamal, A., Alghamdi, H., Alhothali, A.: Arabic rumor detection: A comparative study. Alexandria Engineering Journal 61(12), 12511– 12523 (2022) https://doi.org/10.1016/j.aej.2022.05.029 ↗
  60. Antoun, W., Baly, F., Hajj, H.: Arabert: Transformer-based model for arabic language understanding. In: LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, p. 9 find our card
  61. Antoun, W., Baly, F., Hajj, H.: AraELECTRA: Pre-training text discriminators for Arabic language understanding. In: Proceedings of the Sixth Arabic Natural Language Processing Workshop, pp. 191–195. Association for Computational Linguistics, Kyiv, Ukraine (Virtual) (2021). https://www.aclweb.org/anthology/2021.wanlp-1.20 ↗
  62. Antoun, W., Baly, F., Hajj, H.: AraGPT2: Pre-trained transformer for Arabic language generation. In: Proceedings of the Sixth Arabic Natural Language Processing Workshop, pp. 196–207. Association for Computational Linguistics, Kyiv, Ukraine (Virtual) (2021). https://www.aclweb.org/anthology/2021.wanlp-1.21 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abdulsalam-2023-detecting-suicidality-arabic-tweets-using/