Elamur Research
☰
home / papers / 2016 / The Social Impact of Natural Language Processing

The Social Impact of Natural Language Processing

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P16-2096.pdf

Summary

This position paper argues that NLP needs a field-wide ethics discussion. NLP has moved from largely anonymous corpora such as newswire and novels to social media text written by living people, so its outputs can now affect individuals directly. Until then, institutional ethics review (IRB) was rarely triggered for NLP work. The authors, an NLP researcher and an ethicist, look beyond privacy, which they treat as a general data-science concern, to social justice. They introduce vocabulary from the ethics of technology and organize harms into several categories:

  • Exclusion: demographic bias in the data.
  • Overgeneralization: modeling errors that confirm biases.
  • Topic under- and overexposure: a matter of research design.
  • Dual use: beneficial tools repurposed for harm.

For each category they suggest mitigations, such as rebalancing data, abstaining from low-confidence predictions, and designing research carefully. They note that dual-use risks need collective action from the community rather than individual fixes. The paper matters because practitioners, rather than an alarmist public debate, are best placed to set ethical norms for language technology.

Key points

  • A search of the ACL anthology for "ethic*" returned only three papers. One was a panel and another a book review, so only Couillault et al. (2014) actually addressed ethics.
  • Drawing on work by Jørgensen et al. (2015) as well as Hovy and Søgaard (2015), the authors report that leading models perform worse for young people and ethnic minorities. Proposed fixes include:
    • downsampling the over-represented group,
    • using priors,
    • using demographic statistics as supervision (Mohammady and Culotta, 2014).
  • Against overgeneralization, they recommend dummy labels, error weighting, regularization and confidence thresholds. The guiding test is whether a wrong answer would be worse than giving no answer.
  • In a 2013 Twitter sample, 11 of the 31 most frequent languages had no treebank. The authors also describe English as a typological outlier whose dominance in NLP reinforces itself.
  • Figure 1 tracks ACL title keywords ("grammar" vs. "neural") from 2000 to 2015 to illustrate waves of topic overexposure. Dual-use examples include stylometry that could unmask dissidents and fake-review detection that could equally be used to generate fake reviews.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

201 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 184 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-2096

@inproceedings{hovy2016social,
  title = {{The Social Impact of Natural Language Processing}},
  author = {Hovy, Dirk and Spruit, Shannon L.},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-2096},
  url = {https://doi.org/10.18653/v1/p16-2096}
}

References

66 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asif Agha. 2005. Voice, footing, enregisterment. Journal of linguistic anthropology, pages 38–59. find
  2. Željko Agić, Dirk Hovy, and Anders Søgaard. 2015. If all you have is a bit of the Bible: Learning POS taggers for truly low-resource languages. In Proceedings of the 53rd annual meeting of the ACL. find
  3. Jalal S Alowibdi, Ugo A Buy, and Philip Yu. 2013. Empirical evaluation of profile characteristics for gender classification on twitter. In Machine Learning and Applications (ICMLA), 2013 12th International Conference on, volume 1, pages 365–369. IEEE. find
  4. Ashton Anderson, Dan McFarland, and Dan Jurafsky. 2012. Towards a computational history of the ACL: 1980-2008. In Proceedings of the ACL-2012 Special Workshop on Rediscovering 50 Years of Discoveries, pages 13–21. Association for Computational Linguistics. \#\# Conclusion In this position paper, we ou… find
  5. David Bamman, Brendan O’Connor, and Noah Smith. 2012. Censorship and deletion practices in Chinese social media. First Monday, 17(3). find
  6. For a perspective in a related field see https: //web.eecs.umich.edu/˜kuipers/opinions/ no-military-funding.html Karën Fort, Gilles Adda, and K Bretonnel Cohen. 2011. Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics, 37(2):413–420. find
  7. David Bamman, Chris Dyer, and Noah A. Smith. 2014. Distributed representations of geographically situated language. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 828–834. Proceedings of ACL. find
  8. Victor Galaz, Fredrik Moberg, and Fernanda Torre. 2015. The Biosphere Code Manifesto. http://thebiospherecode.com/index. php/manifesto Retrieved Feb 24, 2016. ↗
  9. Ritwik Banerjee, Song Feng, Seok Jun Kang, and Yejin Choi. 2014. Keystroke patterns as prosody in digital writings: A case study with deceptive reviews and essays. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1469–1473. Association for Comp… find
  10. Daniel G Goldstein and Gerd Gigerenzer. 2002. Models of ecological rationality: the recognition heuristic. Psychological review, 109(1):75. find
  11. Joseph Henrich, Steven J Heine, and Ara Norenzayan. 2010. The weirdest people in the world? Behavioral and brain sciences, 33(2-3):61–83. find
  12. Tom L Beauchamp and James F Childress. 2001. Principles of biomedical ethics. Oxford University Press, USA. find
  13. Pierre Bourdieu and Jean-Claude Passeron. 1990. Reproduction in education, society and culture, volume 4. Sage. find
  14. Dirk Hovy and Anders Johannsen. 2016. Exploring Language Variation Across Europe - A Web-based Tool for Computational Sociolinguistics. In Proceedings of LREC. find
  15. David Bracewell and Marc Tomlinson. 2012. The language of power and its cultural influence. In Proceedings of COLING 2012: Posters, pages 155–164. find
  16. The COLING 2012 Organizing Committee. find
  17. Dirk Hovy and Anders Søgaard. 2015. Tagging performance correlates with author age. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. find
  18. Morgane Ciot, Morgan Sonderegger, and Derek Ruths. 2013. Gender Inference of Twitter Users in Non-English Contexts. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash, pages 18–21. find
  19. Dirk Hovy. 2015. Demographic factors improve classification performance. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. find
  20. Dirk Hovy. 2016. The Enemy in Your Own Camp: How Well Can We Detect Statistically-Generated Fake Reviews–An Adversarial Study. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. find
  21. Alain Couillault, Karën Fort, Gilles Adda, and Hugues Mazancourt (de). 2014. Evaluating corpora documentation with regards to the Ethics and Big Data Charter. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Associati… find
  22. Jeremy Hsu. 2012. Control dangerous AI before it controls us, one expert says. NBC News, March 1. http://www.nbcnews.com/id/46590591/ ns/technology and science-innovation Retrieved Feb 24, 2016. ↗
  23. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In Proceedings of the 49th annual meeting of the ACL. find
  24. Lisa Eadicicco. 2015. Bill Gates: Elon Musk Is Right, We Should All Be Scared Of Artificial Intelligence Wiping Out Humanity. Business Insider, January 28. http://www.businessinsider.com/bill-gates-artificial-intelligence-2015-1 Retrieved Feb 24, 2016. ↗
  25. Oren Etzioni. 2014. Its Time to Intelligently Discuss Artificial Intelligence. Backchannel, December 9 https://backchannel.com/ai-wont-exterminate-us-it-will-empower-us-5b7224735bf3\#.eia6vtimy Retrieved Feb 24, 2016. ↗
  26. Hongzhao Huang, Zhen Wen, Dian Yu, Heng Ji, Yizhou Sun, Jiawei Han, and He Li. 2013. Resolving entity morphs in censored data. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1083–1093. Association for Computational Linguistic… find
  27. Anders Johannsen, Dirk Hovy, and Anders Søgaard. 2015. Cross-lingual syntactic variation over age and gender. In Proceedings of CoNLL. find
  28. Hans Jonas. 1984. The Imperative of Responsibility: Foundations of an Ethics for the Technological Age. Original in German: Prinzip Verantwortung.) Chicago: University of Chicago Press. find
  29. Tommaso Fornaciari and Massimo Poesio. 2014. Identifying fake amazon reviews as learning from crowds. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 279–287. Association for Computational Linguistics. find
  30. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2015. Challenges of studying and processing dialects in social media. In Workshop on Noisy User-generated Text (W-NUT). the American Statistical Association, 58(302):275– Raffi Khatchadourian. 2015. The Dooms-309. day Invention: Will artificial intellig… find
  31. Cathy O’Neil. 2016. The Ethical Data Scientist. Slate, February 4 Adam DI Kramer, Jamie E Guillory, and Jeffrey T Han-http://www.slate.com/articles/ cock. 2014. Experimental evidence of massive-technology/future tense/2016/02/ scale emotional contagion through social networks. how to bring better et… find
  32. Myle Ott, Yejin Choi, Claire Cardie, and T. Jeffrey Hancock. 2011. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 309–319. Association for Computational L… find
  33. Kornel Laskowski. 2010. Modeling norms of turn-taking in multi-party conversation. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 999–1008. Association for Computational Linguistics. find
  34. Wendy Liu and Derek Ruths. 2013. What’s in a name? using first names as features for gender inference in twitter. In Analyzing Microtext: 2013 AAAI Spring Symposium. find
  35. Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch. 2014. The language demographics of amazon mechanical turk. Transactions of the Association of Computational Linguistics – Volume 2, Issue 1, pages 79–92. find
  36. Benjamin Mandel, Aron Culotta, John Boulahanis, Danielle Stark, Bonnie Lewis, and Jeremy Ro-drigue, 2012. Proceedings of the Second Workshop on Language in Social Media, chapter A Demographic Analysis of Online Sentiment during Hurricane Irene, pages 27–36. Association for Computational Linguistics. find
  37. Barbara Plank and Dirk Hovy. 2015. Personality traits on twitterorhow to get 1,500 personality tests in a week. In Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 92–98. find
  38. Tony McEnery. 2002. Ethical and legal issues in corpus construction. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02). European Language Resources Association (ELRA). find
  39. Vinodkumar Prabhakaran and Owen Rambow. 2013. Written dialog and social power: Manifestations of different types of power in dialog behavior. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 216–224. Asian Federation of Natural Language Processing. find
  40. Robert K Merton. 1973. The normative structure of science. The sociology of science: Theoretical and empirical investigations, 267. find
  41. Vinodkumar Prabhakaran, Ashima Arora, and Owen Rambow. 2014. Staying on topic: An indicator of power in political debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1481–1486. Association for Computational Linguistics. find
  42. Sara Mills. 2012. Gender matters: Feminist linguistic analysis. Equinox Pub. find
  43. Ehsan Mohammady and Aron Culotta, 2014. Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media, chapter Using County Demographics to Infer Attributes of Twitter Users, pages 7–16. Association for Computational Linguistics. find
  44. Daniel Preotiuc-Pietro, Vasileios Lampos, and Nikolaos Aletras. 2015a. An analysis of the user occupational class through twitter content. In ACL. find
  45. Daniel Preoţiuc-Pietro, Svitlana Volkova, Vasileios Lampos, Yoram Bachrach, and Nikolaos Aletras. 2015b. Studying user income through language, behaviour and affect in social media. PloS one, 10(9):e0138717. find
  46. Frederick Mosteller and David L Wallace. 1963. Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed federalist papers. Journal of and privacy. Ethics and Information Technology, 17(2):153–163. find
  47. Cornelius Puschmann and Engin Bozdag. 2014. Staking out the unclear ethical terrain of online social experiments. Internet Policy Review, 3(4). find
  48. Oren Tsur, Dan Calacci, and David Lazer. 2015. A frame of mind: Using statistical models for detection of framing and agenda setting campaigns. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Pr… find
  49. Phillip Rogaway. 2015. The moral character of cryptographic work. Technical report, IACR-Cryptology ePrint Archive. find
  50. Sara Rosenthal and Kathleen McKeown. 2011. Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 763– 77… find
  51. Amos Tversky and Daniel Kahneman. 1973. Availability: A heuristic for judging frequency and probability. Cognitive psychology, 5(2):207–232. find
  52. Svitlana Volkova, Theresa Wilson, and David Yarowsky. 2013. Exploring demographic language variations to improve multilingual sentiment analysis in social media. In Proceedings of EMNLP, pages 1815–1827. find
  53. Stuart Russell, Daniel Dewey, Max Tegmark, Janos Kramar, and Richard Mallah. 2015. Research priorities for robust and beneficial artificial intelligence. Technical report, Future of Life Institute. find
  54. Tyler Schnoebelen. 2013. The weirdest languages. Idibon Blog, June 21 http://idibon.com/the-weirdest-languages Retrieved May 17, 2016. ↗
  55. Svitlana Volkova, Glen Coppersmith, and Benjamin Van Durme. 2014. Inferring user political preferences from streaming communications. In Proceedings of the 52nd annual meeting of the ACL, pages 186–196. find
  56. Michael Silverstein. 2003. Indexical order and the dialectics of sociolinguistic life. Language & Communication, 23(3):193–229. find
  57. Svitlana Volkova, Yoram Bachrach, Michael Armstrong, and Vijay Sharma. 2015. Inferring latent user properties from texts published in social media (demo). In Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI), Austin, TX, January. find
  58. Paul Slovic, Melissa L. Finucane, Ellen Peters, and Donald G. MacGregor. 2007. The affect heuristic. European Journal of Operational Research, 177(3):1333 – 1352. find
  59. Anders Søgaard, Željko Agić, Héctor Martı́nez Alonso, Barbara Plank, Bernd Bohnet, and Anders Johannsen. 2015. Inverted indexing for cross-lingual nlp. In Proceedings of the 53rd annual meeting of the ACL. find
  60. Hanna Wallach. 2014. Big Data, Machine Learning, and the Social Sciences: Fairness, Accountability, and Transparency. https://medium.com/@hannawallach/ big-data-machine-learning-and-the-social-sciences-927a8e20460d\#. uhbcr4wa0 Retrieved Feb 24, 2016. ↗
  61. Anders Søgaard. 2011. Data point selection for cross-language adaptation of dependency parsers. In Proceedings of ACL. find
  62. David Yarowsky and Grace Ngai. 2001. Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora. In Proceedings of NAACL. find
  63. Michael Strube. 2015. It is never as simple as it seems: The wide-ranging impacts of ethics violations. Ethical Challenges in the Behavioral and Brain Sciences, page 126. find
  64. Cass R Sunstein. 2004. Precautions against what? the availability heuristic and cross-cultural risk perceptions. U Chicago Law & Economics, Olin Working Paper, (220):04–22. find
  65. Joel Tetreault, Jill Burstein, and Claudia Leacock, 2015. Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, chapter Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguist… find
  66. Jonathan Tse, Dawn E Schrader, Dipayan Ghosh, Tony Liao, and David Lundie. 2015. A bibliometric analysis of privacy and ethics in ieee security find