Elamur Research⌕
☰
home / papers / 2017 / Ethical Research Protocols for Social Media Health Research

Ethical Research Protocols for Social Media Health Research

Summary

Computer scientists increasingly use public social media to study health, but many lack training in human-subjects ethics, and IRBs at different institutions disagree on what such studies require. Benton, Coppersmith and Dredze, writing as practitioners rather than ethicists, offer a practical set of protocol recommendations that newcomers can adapt for an IRB application. They ground the guidance in mental health research, where stigma and lethal outcomes make both risks and benefits especially sharp. Before giving recommendations, they state their own guiding tenets: the aim is societal benefit, interventions can help or harm, classifiers make mistakes, and users often misjudge how public their posts are. The guidelines then address several practical issues, including:

  • whether a study counts as human-subjects research
  • de-identifying data
  • caution about linking accounts across platforms
  • conditions for sharing datasets

They also point to supporting software. The main contribution is a flexible starting skeleton, not a fixed standard. This matters because careless handling of sensitive data can erode public trust and discourage people from seeking treatment.

Key points

  • Regulatory background. The work builds on the Belmont Report (1978) and the proposed modernization of the U.S. Common Rule. The authors argue the proposal widens exemptions but leaves social media researchers' obligations unclear.
  • Mental health as the use case. The authors cite over 40,000 U.S. suicides per year as the cost of insufficient research, and stigma as the cost of insufficient privacy protection.
  • De-identification in practice.
    • The 2015 CLPsych shared task stripped usernames, URLs and most metadata from tweets.
    • The authors recommend the open tool qntfy/deidentify_twitter.
    • In earlier work, Benton et al. (2011) built a model that stripped names, locations, phone numbers and addresses out of breast cancer forum posts.
  • Risks of re-identification. Linked public data can expose identities, as shown by the Netflix challenge data matched against IMDb and the NYC Taxi Dataset de-anonymized using four intersections.
  • Sharing conditions.
    • MIMIC-III requires a human-subjects training certificate before granting access.
    • Section F.2 of Twitter's Developer Policy caps sharing at 50,000 tweets/user objects per downloader per day.

Interesting ideas

  • I1. Write down your research tenets first. Before designing a protocol, list explicit statements about benefits, possible harms, model error and user expectations. These give a consistent basis for later decisions, and the habit transfers to any project handling sensitive data.
  • I2. Treat classifier error as an ethical factor. The authors require researchers to plan for misclassification and possible database breaches, not just to report accuracy. This is a useful reminder for any NLP system whose predictions concern real people.
  • I3. Share data in a form others cannot trace back. Instead of releasing raw posts, which a web search can link to their authors, release opaque feature vectors or texts with tokens replaced by de-identified symbols. Other researchers can still train models on these, which makes it a reusable pattern for sensitive corpora.
  • I4. Make data access depend on matching protections. Recipients of a dataset should commit to the same protective protocols, possibly with proof of ethics training or their own IRB approval, as MIMIC-III does. Any group distributing sensitive annotated corpora could adopt this model.
  • I5. Treat data combinations as a new risk. Two datasets that are each public can become sensitive once linked, for example a public Twitter persona and an anonymous health-forum account. Linking decisions therefore deserve case-by-case review, which matters for geographic, trade or user data as well.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

77 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 74 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1612

@inproceedings{benton2017ethical,
  title = {{Ethical Research Protocols for Social Media Health Research}},
  author = {Benton, Adrian and Coppersmith, Glen and Dredze, Mark},
  booktitle = {Proceedings of the First ACL Workshop on Ethics in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/w17-1612},
  url = {https://doi.org/10.18653/v1/w17-1612}
}

References

31 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adrian Benton, Lyle Ungar, Shawndra Hill, Sean Hennessy, Jun Mao, Annie Chung, Charles E. Leonard, and John H. Holmes. 2011. Identifying potential adverse effects using the web: A new approach to medical hypothesis generation. Journal of biomedical informatics, 44(6):989–996. find
  2. European Parliament and Council of the European Union. 2001. Directive 2001/20/EC. find
  3. James M. Hudson and Amy Bruckman. 2004. “Go away”: participant objections to being studied and the ethics of chatroom research. The Information Society, 20(2):127–139. find
  4. Johan Bollen, Huina Mao, and Xiaojun Zeng. 2011. Twitter mood predicts the stock market. Journal of computational science, 2(1):1–8. find
  5. Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Liwei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, 3. find our card
  6. John D. Burger, John Henderson, George Kim, and Guido Zarrella. 2011. Discriminating gender on twitter. In Empirical Methods in Natural Language Processing (EMNLP), pages 1301–1309. find our card
  7. Fabio Celli, Fabio Pianesi, David Stillwell, and Michal Kosinski. 2013. Workshop on computational personality recognition (shared task). In Workshop on Computational Personality Recognition. find
  8. Miltiadis Kandias, Konstantina Galbogini, Lilian Mitrou, and Dimitris Gritzalis. 2013. Insiders trapped in the mirror reveal themselves in social media. In International Conference on Network and System Security, pages 220–235. find
  9. Mike Conway. 2014. Ethical issues in using Twitter for public health surveillance and research: developing a taxonomy of ethical concepts from the research literature. Journal of Medical Internet Research, 16(12):e290. find
  10. Adam D. I. Kramer, Jamie E. Guillory, and Jeffrey T. Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24):8788–8790. find
  11. Glen Coppersmith, Mark Dredze, Craig Harman, and Kristy Hollingshead. 2015a. From ADHD to SAD: analyzing the language of mental health on Twitter through self-reported diagnoses. In NAACL Workshop on Computational Linguistics and Clinical Psychology. find
  12. Rebecca McKee. 2013. Ethical issues in using social media for health and health care research. Health Policy, 110(2):298–301. find
  13. Glen Coppersmith, Mark Dredze, Craig Harman, Kristy Hollingshead, and Margaret Mitchell. 2015b. find
  14. CLPsych 2015 shared task: Depression and ptsd on Twitter. In Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 31–39. find our card
  15. Jude Mikal, Samantha Hurst, and Mike Conway. 2016. Ethical issues in using Twitter for population-level depression monitoring: a qualitative study. BMC medical ethics, 17(1):1. find
  16. Sally C. Curtin, Margaret Warner, and Holly Hedegaard. 2016. Increase in suicide in the United States, 1999-2014. NCHS data brief, 241:1–8. find
  17. Arvind Narayanan and Vitaly Shmatikov. 2008. Robust de-anonymization of large sparse datasets. In IEEE Symposium on Security and Privacy, pages 111–125. find
  18. Munmun De Choudhury, Emre Kiciman, Mark Dredze, Glen Coppersmith, and Mrinal Kumar. 2016. Discovering shifts to suicidal ideation from mental health content in social media. In Conference on Human Factors in Computing Systems (CHI), pages 2098–2110. find
  19. National Commission. 1978. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research-the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. US Government Printing Office. find
  20. Effy Vayena, Anna Mastroianni, and Jeffrey Kahn. 2013. Caught in the web: informed consent for online health research. Sci Transl Med, 5(173):1–3. find
  21. National Research Council. 2014. Proposed revisions to the common rule for the protection of human subjects in the behavioral and social sciences. National Academies Press. find
  22. Fabian Neuhaus and Timothy Webmoor. 2012. Agile ethics for massified research and visualization. Information, Communication & Society, 15(1):43–65. find
  23. Dan O’Connor. 2013. The apomediated world: regulating research when social media has changed research. Journal of Law, Medicine, and Ethics, 41(2):470–483. find
  24. Kevin A. Padrez, Lyle Ungar, H. Andrew Schwartz, Robert J. Smith, Shawndra Hill, Tadas Antanavicius, Dana M. Brown, Patrick Crutchley, David A. Asch, and Raina M. Merchant. 2015. Linking social media and medical record data: a study of adults presenting to an academic, urban emergency department. Qu… find
  25. Michael J. Paul and Mark Dredze. 2011. You are what you tweet: Analyzing twitter for public health. In International Conference on Weblogs and Social Media (ICWSM), pages 265–272. find
  26. Daniel Preotiuc-Pietro, Johannes Eichstaedt, Gregory Park, Maarten Sap, Laura Smith, Victoria Tobolsky, H. Andrew Schwartz, and Lyle Ungar. 2015. The role of personality, age and gender in tweeting about mental illnesses. In Workshop on Computational Linguistics and Clinical Psychology: From Linguis… find
  27. Daniele Quercia, Michal Kosinski, David Stillwell, and Jon Crowcroft. 2011. Our Twitter profiles, our selves: Predicting personality with Twitter. In IEEE Third International Conference on Privacy, Security, Risk and Trust (PASSAT), pages 180–185. find
  28. Philip Resnik, Anderson Garron, and Rebecca Resnik. 2013. Using topic modeling to improve prediction of neuroticism and depression. In Empirical Methods in Natural Language Processing (EMNLP), pages 1348–1353. find
  29. H. Andrew Schwartz, Johannes C. Eichstaedt, Margaret L. Kern, Lukasz Dziurzynski, Richard E. Lucas, Megha Agrawal, Gregory J. Park, Shrinidhi K. Lakshmikanth, Sneha Jha, Martin E. P. Seligman, and Lyle H. Ungar. 2013. Characterizing geographic variation in well-being using tweets. In International C… find
  30. Andranik Tumasjan, Timm O. Sprenger, Philipp G. Sandner, and Isabell M. Welpe. 2011. Election forecasts with twitter: How 140 characters reflect the political landscape. Social science computer review, 29(4):402–418. find
  31. US Department of HHS. 2009. Code of federal regulations. title 45. Public Welfare CFR, 46. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/benton-2017-ethical-research-protocols-social-media/