Elamur Research⌕
☰
home / papers / 2022 / The Ghost in the Machine has an American accent: value conflict in GPT-3

The Ghost in the Machine has an American accent: value conflict in GPT-3

Paper · 2022 · Rebecca L Johnson, Giada Pistilli, Natalia Menedez-Gonzalez et al.

Summary

Large language models absorb the cultural values of the people whose writing fills their training data. Aligning them with "human values" is therefore hard, because those values differ across societies and sometimes clash legitimately. The authors study this problem for GPT-3.

First, they compare the composition of its training corpus, dominated by English web text, with global distributions of languages and internet access. They also compare it with national value profiles from the World Values Survey (WVS). Second, they probe the model in an exploratory, qualitative way. GPT-3 receives value-laden passages from several languages and countries, including some whose stance diverges from prevailing US opinion. The generated outputs are then checked for shifts in the values expressed. When such shifts occurred, they often moved toward positions the WVS reports as dominant in the United States.

To interpret these "value mutations", the paper draws on moral value pluralism (MVP), especially Nagel's typology of conflicts and values. The authors argue this lens can help developers spot cultural skew and guide fine-tuning toward more pluralist alignment.

Key points

  • About 60% of GPT-3's training data comes from a filtered CommonCrawl. The remainder includes a scraped-links dataset, two books corpora and English-language Wikipedia, and over 93% of the total is English.
  • The method is stress testing with value-rich input texts, judged against WVS national profiles. Altered values in outputs frequently matched dominant US views, which gives the paper its "American accent" title.
  • Following Nagel, the paper separates strong conflicts (values that condemn each other) from weak ones (incompatibilities a community can tolerate). It uses five value categories: obligations, rights, utility, perfectionist ends and private commitments.
  • The authors propose a sixth category: fair sharing of collective responsibility for global issues such as the environment and sustainability.
  • They recommend anchoring LLMs to basic rights, for example the Universal Declaration of Human Rights, while accommodating conflicting cultural value systems. They also call for an MVP roadmap for fine-tuning and careful human-in-the-loop design. The study is framed as exploratory "slow research" without a quantitative benchmark.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Rebecca L Johnson, Giada Pistilli, Natalia Menedez-Gonzalez, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene and Donald Jay Bertulfo. 2022. The Ghost in the Machine has an American accent: value conflict in GPT-3. arXiv:2203.07785.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2203.07785 · arXiv 2203.07785

@misc{johnson2022ghost,
  title = {{The Ghost in the Machine has an American accent: value conflict in GPT-3}},
  author = {Rebecca L Johnson, Giada Pistilli, Natalia Menedez-Gonzalez, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene and Donald Jay Bertulfo},
  year = {2022},
  doi = {10.48550/arxiv.2203.07785},
  eprint = {2203.07785},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2203.07785}
}

References

81 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [1] Floridi, L. and Chiriatti, M. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30, 4 (2020), 681-694. find
  2. [2] Abid, A., Farooqi, M. and Zou, J. Persistent anti-muslim bias in large language models. arXiv:2101.05783 (2021).
  3. [3] Solaiman, I. and Dennison, C. Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets. arXiv:2106.10328 (2021).
  4. [4] Tamkin, A., Brundage, M., Clark, J. and Ganguli, D. Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. arXiv:2102.02503 (2021).
  5. [5] Hofstede, G. Culture's consequences: Comparing values, behaviors, institutions and organizations across nations. Sage publications, 2001. find
  6. [6] Rokeach, M. Understanding human values. Simon and Schuster, 2008. find
  7. [7] Jonkers, P. How to Respond to Conflicts Over Value Pluralism? Journal of Nationalism, Memory, and Language-Politics (2019). find
  8. [8] Yudkowsky, E. The AI alignment problem: why it is hard, and where to start. Symbolic Systems Distinguished Speaker (2016). find
  9. [9] Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A. and Brunskill, E. On the opportunities and risks of foundation models. arXiv:2108.07258 (2021).
  10. [10] Hume, D. A treatise of human nature. Clarendon Press, 1896. find
  11. [11] Gabriel, I. Artificial intelligence, values, and alignment. Minds and Machines, 30, 3 (2020), 411-437. find
  12. [12] Russell, S. Human-compatible artificial intelligence. Human-Like Machine Intelligence, Oxford University Press Oxford, 2021. find
  13. [13] Wiener, N. Some Moral and Technical Consequences of Automation: As machines learn they may develop unforeseen strategies at rates that baffle their programmers. Science, 131, 3410 (1960), 1355-1358. find
  14. [14] Habermas, J. Moral consciousness and communicative action. MIT press, 1990. find
  15. [15] Barthes, R. Elements of Semiology. 1964. find
  16. New York: Hill & Wang (1967), 41-47. find
  17. [16] Peeters, B. Language and cultural values. International Journal of Language and Culture, 2, 2 (2015), 133-141. find
  18. [18] Stearn, R. L. Technology and world trade : proceedings of a symposium, November 16-17, 1966. U.S. Department of Commerce, National Bureau of Standards, Washington, 1967. find
  19. [19] Lakoff, G. and Johnson, M. Metaphors we live by. University of Chicago press, 2008. find
  20. [20] Peeters, B. Tall poppies and egalitarianism in Australian discourse: From key word to cultural value. English worldwide, 25, 1 (2004), 1-25. find
  21. [21] Nieminen, T. The nail that sticks out: the practice of individuality in the East Asian classroom (2015). find
  22. [22] Walker, J., Tepper, S. J. and Gilovich, T. People are more tolerant of inequality when it is expressed in terms of individuals rather than groups at the top. Proceedings of the National Academy of Sciences, 118, 43 (2021). find
  23. [23] Stephens, G. J., Silbert, L. J. and Hasson, U. Speaker– listener neural coupling underlies successful communication. Proceedings of the National Academy of Sciences, 107, 32 (2010), 14425-14430. find
  24. [24] Clark, H. H. Using language. Cambridge university press, 1996. find
  25. [25] Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. and Kalai, A. T. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29 (2016).
  26. [26] Izzidien, A. Word vector embeddings hold social ontological relations capable of reflecting meaningful fairness assessments. AI & SOCIETY, 37, 1 (2022/03/01 2022), 299-318. find
  27. [27] Topal, M. O., Bas, A. and van Heerden, I. Exploring transformers in natural language generation: Gpt, bert, and xlnet. arXiv:2102.08036 (2021). — [arXiv:2102.08036](https://arxiv.org/abs/2102.08036) find
  28. [28] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. and Polosukhin, I. Attention is all you need. In Proceedings of the Advances in neural information processing systems, 2017
  29. [29] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. and Galstyan, A. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54, 6 (2021), 1-35.
  30. [30] Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B. and Kasirzadeh, A. Ethical and social risks of harm from Language Models. arXiv:2112.04359 (2021).
  31. [31] Hymes, D. Models of the interaction of language and social life: toward a descriptive theory. Intercultural discourse and communication: The essential readings, 1 (2005), 4-16. find
  32. [32] Fishman, J. What Do You Lose When You Lose Your Language? (1996). find
  33. [17] Lakoff, G. Women, fire, and dangerous things : what categories reveal about the mind. University of Chicago Press, Chicago, 1987. find
  34. [33] Dewey, J. The logic of judgments of practise. The Journal of philosophy, psychology and scientific methods, 12, 19 (1915), 505-523. find
  35. [34] Jean, S. Les valeurs du temps présent: une enquête européenne. Paris: Presses universitaires de France (1983). find
  36. [35] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G. and Askell, A. Language models are few-shot learners. arXiv:2005.14165 (2020). the Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 2021
  37. [48] Lucy, L. and Bamman, D. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the The Third Workshop on Narrative Understanding, 2021
  38. [49] Fazelpour, S. and De-Arteaga, M. Diversity in Sociotechnical Machine Learning Systems. arXiv:2107.09163 (2021). — [arXiv:2107.09163](https://arxiv.org/abs/2107.09163) find
  39. [36] CommonCrawl, Distribution of Languages. 2021, https://commoncrawl.github.io/cc-crawl-statistics/plots/languages ↗
  40. [50] Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L. and Du, Y. LaMDA: Language Models for Dialog Applications. arXiv:2201.08239 (2022).
  41. [37] Eberhard David M., S. G. F., Fenning Charles D.,, Summary by language size. Ethnologue Languages of the World, 2019, http://www.ethnologue.com ↗
  42. [51] Galston, W. A. Liberal pluralism: The implications of value pluralism for political theory and practice. Cambridge University Press, 2002. find
  43. [38] Bender, E. M., Gebru, T., McMillan-Major, A. and Shmitchell, S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the The 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021 find
  44. [52] Berlin, I. Four essays on liberty. (1969). find
  45. [39] Johnson, J., Internet usage worldwide - statistics & facts. 2021, https://www.statista.com/topics/1145/internet-usage-worldwide/ ↗
  46. [54] McSweeney, B. The essentials of scholarship: A reply to Geert Hofstede. Human relations, 55, 11 (2002), 1363-1372. find
  47. [40] International Telecommunication Union, Individuals using the Internet (% of population) - Sub-Saharan Africa. The World Bank, — StereoSet: Measuring stereotypical bias in pretrained language models https://data.worldbank.org/indicator/IT.NET.USER.ZS?locati ons=ZG [41] Nadeem, M., Bethke, A. and Reddy, S. Stereoset: Measuring stereotypical bias in pretrained languag… ↗
  48. [42] Lu, K., Mardziel, P., Wu, F., Amancharla, P. and Datta, A. Gender bias in neural natural language processing. Logic, Language, and Security, Springer, 2020.
  49. [43] Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y. and Shieber, S. M. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In Proceedings of the NeurIPS, 2020 find
  50. [44] Russell, S. Human compatible: Artificial intelligence and the problem of control. Penguin, 2019. find
  51. [45] Christian, B. The Alignment Problem: How Can Machines Learn Human Values? Atlantic Books, 2021. find
  52. [46] Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K.-W. and Gupta, R. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the The 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021 find
  53. [47] Guo, W. and Caliskan, A. Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases. In Proceedings of [53] Tausch, A. Hofstede, Inglehart and Beyond. New Directions in Empirical Global Value Research. SSRN (2015). find
  54. [55] Schwartz, S. H. and Bardi, A. Value hierarchies across cultures: Taking a similarities perspective. Journal of cross-cultural Psychology, 32, 3 (2001), 268-290. find
  55. [56] World Values Survey, WVS wave 7. 2022, https://www.worldvaluessurvey.org/WVSContents.jsp ↗
  56. [57] Weber, M. and Kalberg, S. The Protestant ethic and the spirit of capitalism. Routledge, 2013. find
  57. [58] Gray, M. L. and Suri, S. Ghost work: how to stop Silicon Valley from building a new global underclass. Eamon Dolan Books, 2019. find
  58. [59] Davani, A. M., Díaz, M. and Prabhakaran, V. Dealing with disagreements: Looking beyond the majority vote in subjective annotations. arXiv:2110.05719 (2021).
  59. [60] Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M. and Dean, J. Carbon emissions and large neural network training. arXiv:2104.10350 (2021).
  60. [61] Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M. and Le, Q. V. Finetuned language models are zero-shot learners. arXiv:2109.01652 (2021).
  61. [62] Nikolich, A. and Puchkova, A. Fine-tuning GPT-3 for Russian Text Summarization. arXiv:2108.03502 (2021). — [arXiv:2108.03502](https://arxiv.org/abs/2108.03502) find
  62. [63] Moradi, M., Blagec, K., Haberl, F. and Samwald, M. GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain. arXiv:2109.02555 (2021). — [arXiv:2109.02555](https://arxiv.org/abs/2109.02555) find
  63. [64] Kirstain, Y., Lewis, P., Riedel, S. and Levy, O. A Few More Examples May Be Worth Billions of Parameters. arXiv:2110.04374 (2021). find
  64. [65] Reynolds, L. and McDonell, K. Prompt programming for large language models: Beyond the few-shot paradigm. In Proceedings of the The 2021 CHI Conference on Human Factors in Computing Systems., 2021
  65. [66] Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R. and Young, S. Scaling language models: Methods, analysis & insights from training gopher. arXiv:2112.11446 (2021). find
  66. [67] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K. and Ray, A. InstructGPT: Training language models to follow instructions with human feedback. https://github.com/openai/following-instructions-human-feedback (2022). ↗
  67. [68] Prabhakaran, V., Davani, A. M. and Díaz, M. On releasing Annotator-Level labels and information in datasets. arXiv:2110.05699 (2021).
  68. [69] Newman, J. and Head, B. The national context of wicked problems: Comparing policies on gun violence in the US, Canada, and Australia. Journal of comparative policy analysis: research and practice, 19, 1 (2017), 40-53. find
  69. [79] Freedman, J. Secularism as a barrier to integration? The French dilemma. International Migration, 42, 3 (2004), 5-27. find
  70. [80] United Nations General Assembly Convention on the elimination of all forms of discrimination against women. 2006, https://www.ohchr.org/en/hrbodies/cedaw/pages/cedawindex. aspx [81] Malcolm, X., Baldwin, J. and McCummins, L. The ballot or the bullet. ↗
  71. Paul Winley Records, 1987. find
  72. [82] Tarana Burke, Power of Women Speech. Iowa State University, 2018, https://awpc.cattcenter.iastate.edu/2018/09/26/full-power-of-women-speech-april-13-2018/ ↗
  73. [83] UNESCO Draft text of the recommendation on the ethics of artificial intelligence., 2021, https://unesdoc.unesco.org/ark:/48223/pf0000377897 ↗
  74. [84] Nagel, T. Mortal questions. Cambridge University Press, Cambridge [Eng.] ;, 1979. find
  75. [85] Nagel, T. The Fragmentation of Value. Mortal Questions, Cambridge University Press, Cambridge, 2012. find
  76. [70] Small Arms Survey, Global Fireams Holdings. https://www.smallarmssurvey.org/database/global-firearms-holdings [71] Research, E. The Essential Report. 2016. ↗
  77. [72] Ipsos. What do you think is more important?...To protect the right of Americans to own guns or control gun ownership? , Pew Research Center for the People & the Press, U.S.A, 2019. find
  78. [73] Australasian Police Ministers’ COUNCIL (APMC) National Firearms Agreement. 1996, [74] De Beauvoir, S. “Introduction” to The Second Sex. na, 1997. find
  79. [75] Jones, M., \#IWD2019: Perceptions of violence against women in France and the United States. 2019, https://www.ipsos.com/en/iwd2019-perceptions-violence-against-women-france-and-united-states ↗
  80. [76] Merkel, A. Sommerpressekonferenz von Bundeskanzlerin Merkel. Thema: Aktuelle Themen der Innen-und Außenpolitik. August, 2015, [77] Stasi, B. Rapport au Président de la République. Commission de réflexion sur l'application du principe de laïcité dans la republique., France, 2003. find
  81. [78] Hauser, J. Education, secularism, and illiberalism: Marginalisation of Muslims by the French state. French Cultural Studies, 32, 2 (2021), 149-162. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/johnson-2022-ghost-machine-has-american-accent/