Elamur Research⌕
☰
home / papers / 2020 / CEASE, a Corpus of Emotion Annotated Suicide notes in English

CEASE, a Corpus of Emotion Annotated Suicide notes in English

Paper · 2020 · Soumitra Ghosh, Asif Ekbal and Pushpak Bhattacharyya

Summary

Research on emotion in suicide notes had stalled because the only fine-grained benchmark, the i2b2 2011 shared-task corpus, is no longer distributed. The authors address this by building CEASE, a public English corpus of real suicide notes. It contains 2393 sentences from roughly 205 notes. The notes came from two places: online news outlets and blogs, and the book "...Or Not to Be" by Marc Etkind. Scanned pages were turned into text with OCR, and personal details were masked with typed placeholder tags. Each sentence is read in the context of its whole note and given one primary label. The label set reuses the 15 classes of the i2b2 scheme: positive emotions, negative emotions, and two neutral classes, information and instructions. As benchmarks, the authors train CNN, GRU and LSTM classifiers and combine them into ensembles. The best ensemble reaches 60.17% test accuracy and 60.32% cross-validation accuracy, beating feature-based machine-learning baselines. The resource matters because data in this sensitive domain is scarce, and it supports work on mental-health assessment and suicide prevention.

Key points

  • CEASE has 2393 sentences from about 205 real suicide notes, with one of 15 fine-grained labels per sentence. The labels follow the i2b2 2011 scheme: 6 positive, 7 negative and 2 neutral classes.
  • Collection ran in two phases:
    • Phase one gathered notes published by news sites, e-newspapers and blogs, avoiding unverifiable images. Of these, 31 notes were anonymous.
    • Phase two used Etkind's 1997 book, which held 138 notes, 38 of them anonymous. Duplicates of phase-one notes were dropped.
  • Roughly 58% of training sentences carry a neutral label (information or instructions). This leaves little data for the 13 emotional classes.
  • The base models are CNN, GRU and LSTM, combined into two ensembles. Results are 60.17% test accuracy and 60.32% cross-validation accuracy.
  • The original i2b2 corpus had 900 multi-labelled notes, split 600 for training and 300 for testing. The authors plan to add per-sentence emotion intensity scores for multitask learning.

Interesting ideas

  • I1. The label scheme puts two non-emotional, function-oriented classes (information, instructions) next to the emotions, so that a sentence's role is not forced into an affect category it lacks; this keeps the label space honest for documents that mix feeling with practical directives.
  • I2. Each sentence is labelled while reading the complete document, yet gets only one "primary" label, which trades nuance for clean, comparable annotation.
  • I3. Personal data is replaced by typed placeholders such as ADDRESS or ORGANIZATION rather than deleted, which protects identities while keeping the entity type available to models.
  • I4. To guard against doctored or fake texts, the authors accepted only notes from published, traceable outlets instead of crawling arbitrary images.
  • I5. Several architecturally different classifiers (convolutional and recurrent) are combined, and results are reported on both a held-out test set and cross-validation, which is a sensible check on a small corpus.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could tag each clause both with a speech act (informing vs. instructing) and with an optional affect frame, so an "instruction" is never mistaken for an emotional statement.
  • I2 (frames): a clause-level frame annotator could use document context but commit to a single dominant reading, keeping secondary readings as explainable alternatives.
  • I3 (world): when building the world of a text from sensitive inputs, placeholder entities could keep their ontology type, so states, holdings and event order still resolve without exposing real names.
  • I4 (learn): silver-data harvesting could weight or filter raw text by source provenance before witness parsers vote, lowering the noise that reaches training.
  • I5 (learn): this mirrors agreement among witness parsers, and pairing held-out and cross-validated scores could stabilise evaluation on small hand-checked treebank slices.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology BombayMumbai · India22 works · 2,018 citations🎓Indian Institute of Technology PatnaPatna · India10 works · 262 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Psychology · 798 works Social Psychology · 129 works Clinical Psychology · 10 works

🧭 Topics 2 English · 1,373 works Sentiment & emotion · 608 works

Cite

Cite as Soumitra Ghosh, Asif Ekbal and Pushpak Bhattacharyya. 2020. CEASE, a Corpus of Emotion Annotated Suicide notes in English. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ghosh2020cease,
  title = {{CEASE, a Corpus of Emotion Annotated Suicide notes in English}},
  author = {Soumitra Ghosh, Asif Ekbal and Pushpak Bhattacharyya},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.201/}
}

References

29 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akhtar, M. S., Kumar, A., Ekbal, A., and Bhattacharyya, P. (2016). A hybrid deep learning architecture for sentiment analysis. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 482–493. find
  2. Akhtar, M. S., Gupta, D., Ekbal, A., and Bhattacharyya, P. (2017). Feature selection and ensemble construction: A two-step method for aspect based sentiment analysis. Knowledge-Based Systems, 125:116–135. ↗
  3. Aman, S. and Szpakowicz, S. (2007). Identifying expressions of emotion in text. In International Conference on Text, Speech and Dialogue, pages 196–205. Springer. find
  4. Canetto, S. S. and Lester, D. (1995). The epidemiology of women’s suicidal behavior. Springer Publishing Co. find
  5. Cho, K., Van Merriënboer, B., Bahdanau, D., and Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259. ↗ our card
  6. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. ↗
  7. Duch, W., Matykiewicz, P., and Pestian, J. (2008). Neurolinguistic approach to natural language processing with applications to medical text analysis. Neural Networks, 21(10):1500–1510. ↗
  8. Ekman, P. (1992). An argument for basic emotions. Cognition & emotion, 6(3-4):169–200. ↗ our card
  9. Etkind, M. (1997). –or Not to be: A Collection of Suicide Notes. Riverhead Books. find
  10. Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 315–323. find
  11. Ho, T., Yip, P. S., Chiu, C., and Halliday, P. (1998). Suicide notes: what do they tell us? Acta Psychiatrica Scandinavica, 98(6):467–473. ↗
  12. Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  13. Hutto, C. J. and Gilbert, E. (2014). Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Eighth international AAAI conference on weblogs and social media. find
  14. Joiner, T. E., Pettit, J. W., Walker, R. L., Voelz, Z. R., Cruz, J., Rudd, M. D., and Lester, D. (2002). Perceived burdensomeness and suicidality: Two studies on the suicide notes of those attempting and those completing suicide. Journal of Social and Clinical Psychology, 21(5):531– 545. ↗
  15. Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. ↗ our card
  16. Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  17. Neviarouskaya, A., Prendinger, H., and Ishizuka, M. (2011). Affect analysis model: novel rule-based approach to affect sensing from text. Natural Language Engineering, 17(1):95–135. ↗
  18. Nielsen, F. Å. (2011). A new anew: Evaluation of a word list for sentiment analysis in microblogs. arXiv preprint arXiv:1103.2903. ↗
  19. Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532– 1543. our card
  20. Pestian, J. and Matykiewicz, P. (2008). Classification of suicide notes using natural language processing. Proceedings of ACL Bio NLP, 967. find
  21. Pestian, J. P., Matykiewicz, P., and Grupp-Phelan, J. (2008). Using natural language processing to classify suicide notes. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing, pages 96–97. Association for Computational Linguistics. find
  22. Pestian, J., Nasrallah, H., Matykiewicz, P., Bennett, A., and Leenaars, A. (2010). Suicide note classification using natural language processing: A content analysis. Biomedical informatics insights, 3:BII–S4706. ↗
  23. Pestian, J. P., Matykiewicz, P., Linn-Gust, M., South, B., Uzuner, O., Wiebe, J., Cohen, K. B., Hurdle, J., and Brew, C. (2012). Sentiment analysis of suicide notes: A shared task. Biomedical informatics insights, 5:BII– S9042. ↗
  24. Plutchik, R. (2001). The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American scientist, 89(4):344–350. find
  25. Shneidman, E. S. (1973). Suicide notes reconsidered. Psychiatry, 36(4):379–394. ↗
  26. Singhal, P. and Bhattacharyya, P. (2016). Borrow a little from your rich cousin: Using embeddings and polarities of english words for multilingual sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3053–3062. find
  27. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958. find
  28. Xu, Y., Wang, Y., Liu, J., Tu, Z., Sun, J.-T., Tsujii, J., and Chang, E. (2012). Suicide note sentiment classification: a supervised approach augmented by web data. Biomedical informatics insights, 5:BII–S8956. ↗
  29. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 1480–1489. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ghosh-2020-cease-corpus-emotion-annotated-suicide/