Elamur Research⌕
☰
home / papers / 2022 / Social Simulacra: Creating Populated Prototypes for Social Computing Systems

Social Simulacra: Creating Populated Prototypes for Social Computing Systems

Paper · 2022 · Joon Sung Park, Lindsay Popowski, Carrie J. Cai et al.

Summary

People who design online communities usually test their ideas with a small group of recruited users. Many problems, such as trolling, unclear norms or off-topic content, appear only once a space has many members. The authors propose social simulacra, a prototyping approach that uses GPT-3 to fill a planned community with synthetic members and their conversations. The designer provides a community goal, rules and a few seed personas. A chain of prompts expands these into a large pool of personas and then writes posts and replies that fit the design, including hostile ones. The technique is built into SimReddit, a tool for prototyping subreddits that also lets designers test "what if" moderation actions or replies. In a technical test on 50 recently created subreddits, people trying to tell real threads from generated ones performed close to chance. In a study, 16 social computing designers found use cases and abuse scenarios they had not anticipated, and they revised their rules and goal statements as a result. The approach offers a cheap way to probe failure modes before launch, but it does not replace engaging with real users.

Key points

  • The pipeline uses prompt chains over GPT-3. First it generates thousands of personas from designer-provided seeds. Then it generates posts and replies conditioned on the community goal, rules and moderator interventions.
  • SimReddit is the reference tool. Changing the rules visibly shifts the generated behavior; for example, an explicit norm of constructive feedback reduced troll replies.
  • Technical evaluation: 50 subreddits created after GPT-3's release were regenerated from their goal and rules alone. Participants misjudged on average 41% (std=10) of real-versus-generated pairs.
  • Designer study (N=16): participants discovered unplanned positive uses and unexpected threats, such as Russian trolls in an international affairs community.
  • Limitations noted by the authors:
    • GPT-3 prompts are capped at roughly 8,000 characters.
    • The system handles text only and English only.
    • Biases may be inherited from the model.
    • The tool carries a risk of misuse for astroturfing, so the authors recommend access only for vetted designers and centralized logging of generated content.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States492 works · 440,462 citations🎓Stanford UniversityStanford · United States506 works · 330,052 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksInformation Systems · 1,730 worksHuman-Computer Interaction · 160 works

Cite

Cite as Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang and Michael S. Bernstein. 2022. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. arXiv:2208.04024.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2208.04024 · arXiv 2208.04024

@misc{park2022social,
  title = {{Social Simulacra: Creating Populated Prototypes for Social Computing Systems}},
  author = {Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang and Michael S. Bernstein},
  year = {2022},
  doi = {10.48550/arxiv.2208.04024},
  eprint = {2208.04024},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2208.04024}
}

References

86 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mark S. Ackerman. 2000. The Intellectual Challenge of CSCW: The Gap between Social Requirements and Technical Feasibility. Human-Computer Interaction 15, 2 (sep 2000), 179–203. https://doi.org/10.1207/S15327051HCI1523_5 — [doi:10.1207/s15327051hci1523_5](https://doi.org/10.1207/s15327051hci1523_5) ↗
  2. Yavuz Akbulut, Yusuf Levent Sahin, and Bahadir Eristi. 2010. Cyberbullying victimization among Turkish online social utility members. Journal of Educational Technology & Society 13, 4 (2010), 192–201. find
  3. Michel Beaudouin-Lafon and Wendy Mackay. 2002. Prototyping Tools and Techniques. L. Erlbaum Associates Inc., USA, 1006–1031. find
  4. Michael S. Bernstein, Mark S. Ackerman, Ed H. Chi, and Robert C. Miller. 2011. The Trouble with Social Computing Systems Research. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA ’11). Association for Computing Machinery, New York, NY, USA, 389–398… — [doi:10.1145/1979742.1979618](https://doi.org/10.1145/1979742.1979618) find
  5. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  7. Marion Buchenau and Jane Fulton Suri. 2000. Experience prototyping. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques. 424–433. find
  8. Zoya Bylinskii, Nam Wook Kim, Peter O’Donovan, Sami Alsheikh, Spandan Madan, Hanspeter Pfister, Fredo Durand, Bryan Russell, and Aaron Hertzmann. 2017. Learning visual importance for graphic designs and data visualizations. In Proceedings of the 30th Annual ACM symposium on user interface software a… find
  9. Stevie Chancellor, Zhiyuan (Jerry) Lin, and Munmun De Choudhury. 2016. "This Post Will Just Get Taken Down": Characterizing Removed Pro-Eating Disorder Social Media Content. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Associa… find
  10. Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can’t stay here: The efficacy of reddit’s 2015 ban examined through hate speech. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1–22. find
  11. Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet’s Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 32… — [doi:10.1145/3274301](https://doi.org/10.1145/3274301) find
  12. Eshwar Chandrasekharan, Mattia Samory, Anirudh Srinivasan, and Eric Gilbert. 2017. The bag of communities: Identifying abusive behavior online with preexisting internet data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 3175–3187. find
  13. Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSC… find
  14. Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial behavior in online discussion communities. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 9. 61–70. find
  15. Joan Donovan. 2019. How hate groups’ secret sound system works. The Atlantic 17 (2019). https://doi.org/10.1145/1188913.1188915 — [doi:10.1145/1188913.1188915](https://doi.org/10.1145/1188913.1188915) ↗
  16. Daniel Drew, Julie L Newcomb, William McGrath, Filip Maksimovic, David Mellis, and Björn Hartmann. 2016. The toastboard: Ubiquitous instrumentation and automated checking of breadboarded circuits. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 677–686. find
  17. Casey Fiesler, Joshua McCann, Kyle Frye, Jed R Brubaker, et al. 2018. Reddit rules! characterizing an ecosystem of governance. In Twelfth International AAAI Conference on Web and Social Media. find
  18. Susan T Fiske. 2022. Twitter manipulates your feed: Ethical considerations. find
  19. Catherine Flick. 2016. Informed consent and the Facebook emotional manipulation study. Research Ethics 12, 1 (2016), 14–28. find
  20. Bill Gaver, Tony Dunne, and Elena Pacenti. 1999. Design: cultural probes. interactions 6, 1 (1999), 21–29. find
  21. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. CoRR abs/2009.11462 (2020). arXiv:2009.11462 https://arxiv.org/abs/2009.11462 ↗
  22. Catherine Grevet and Eric Gilbert. 2015. Piggyback prototyping: Using existing, large-scale social computing systems to prototype new ones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 4047–4056. find
  23. Jonathan Grudin. 1994. Groupware and social dynamics: Eight challenges for developers. Commun. ACM 37, 1 (1994), 92–105. find
  24. Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013. The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist 57, 5 (2013), 664–688. find
  25. Blake Hallinan, Jed R Brubaker, and Casey Fiesler. 2020. Unexpected expectations: Public reaction to the Facebook emotional contagion study. New Media & Society 22, 6 (2020), 1076–1094. find
  26. Claire Hardaker. 2010. Trolling in asynchronous computer-mediated communication: From user discussions to academic definitions. Journal of Politeness Research 6 (2010), 215J242. find
  27. Björn Hartmann, Leith Abdulla, Manas Mittal, and Scott R Klemmer. 2007. Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. In Proceedings of the SIGCHI conference on Human factors in computing systems. 145–154. find
  28. Björn Hartmann, Scott R Klemmer, Michael Bernstein, Leith Abdulla, Brandon Burr, Avi Robinson-Mosher, and Jennifer Gee. 2006. Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of the 19th annual ACM symposium on User interface software and technology. 299–… find
  29. Björn Hartmann, Meredith Ringel Morris, Hrvoje Benko, and Andrew D. Wilson. 2009. Augmenting Interactive Tables with Mice & Keyboards. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (Victoria, BC, Canada) (UIST ’09). ACM, New York, NY, USA, 149–152. https:… — [doi:10.1145/1622176.1622204](https://doi.org/10.1145/1622176.1622204) find
  30. Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R. Klemmer. 2008. Design as Exploration: Creating Interface Alternatives through Parallel Authoring and Runtime Tuning. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (Monterey, CA, USA) (UIST… — [doi:10.1145/1449715.1449732](https://doi.org/10.1145/1449715.1449732) find
  31. Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R. Klemmer. 2008. Design as Exploration: Creating Interface Alternatives through Parallel Authoring and Runtime Tuning. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (Monterey, CA, USA) (UIST… — [doi:10.1145/1449715.1449732](https://doi.org/10.1145/1449715.1449732) find
  32. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020).
  33. Stephanie Houde and Charles Hill. 1997. What do prototypes prototype? In Handbook of human-computer interaction (second ed.). Elsevier, 367–381. find
  34. Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. 2022. Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents. arXiv preprint arXiv:2201.07207 (2022).
  35. Rohan Jagtap. 2021. GPT-3 Explained. Retrieved March 2, 2005 from https://towardsdatascience.com/gpt-3-explained-19e5f2bd3288 ↗
  36. Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022. PromptMaker: Prompt-based Prototyping with Large Language Models. In CHI ’22 Extended Abstracts on Human Factors in Computing Systems. find
  37. Ellen Jiang, Edwin Toh, Alejandra Molina, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2021. Genline and genform: Two tools for interacting with generative language models in a code editor. In The Adjunct Publication of the 34th Annual ACM Symposium on User Interface Software and Technology. 145… find
  38. Joseph M Kayany. 1998. Contexts of uninhibited online behavior: Flaming in social newsgroups on Usenet. Journal of the American Society for Information Science 49, 12 (1998), 1135–1141. find
  39. John F Kelley. 1983. An empirical methodology for writing user-friendly natural language computer applications. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 193–196. find
  40. John F Kelley. 1984. An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems (TOIS) 2, 1 (1984), 26–41. find
  41. Charles Kiene, Andrés Monroy-Hernández, and Benjamin Mako Hill. 2016. Surviving an" Eternal September" How an Online Community Managed a Surge of Newcomers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1152–1156. find
  42. Aniket Kittur, Bongwon Suh, Bryan A. Pendleton, and Ed H. Chi. 2007. He Says, She Says: Conflict and Coordination in Wikipedia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA,… — [doi:10.1145/1240624.1240698](https://doi.org/10.1145/1240624.1240698) find
  43. Ron Kohavi, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, and Ya Xu. 2012. Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained (KDD ’12). Association for Computing Machinery, New York, NY, USA, 786–794. https://doi.org/10.1145/2339530.2339653 — [doi:10.1145/2339530.2339653](https://doi.org/10.1145/2339530.2339653) ↗
  44. Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences 111, 24 (2014), 8788–8790. find
  45. Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design. MIT Press. find
  46. James A. Landay. 1996. SILK: Sketching Interfaces like Krazy. In Conference Companion on Human Factors in Computing Systems (Vancouver, British Columbia, Canada) (CHI ’96). Association for Computing Machinery, New York, NY, USA, 398–399. https://doi.org/10.1145/257089.257396 — [doi:10.1145/257089.257396](https://doi.org/10.1145/257089.257396) ↗
  47. Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning for negotiation dialogues. arXiv preprint arXiv:1706.05125 (2017). — [arXiv:1706.05125](https://arxiv.org/abs/1706.05125) find
  48. Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2021. Generated knowledge prompting for commonsense reasoning. arXiv preprint arXiv:2110.08387 (2021). find
  49. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021).
  50. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786 (2021).
  51. J. Marks, B. Andalman, P. A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, H. Pfister, W. Ruml, K. Ryall, J. Seims, and S. Shieber. 1997. Design Galleries: A General Approach to Setting Parameters for Computer Graphics and Animation. In Proceedings of the 24th Annual Conference… — [doi:10.1145/258734.258887](https://doi.org/10.1145/258734.258887) find
  52. M. Lynne Markus. 1987. Toward a “Critical Mass” Theory of Interactive Media. Communication Research 14 (1987), 491 – 511. find
  53. Nathan J. Matias. 2019. Preventing harassment and increasing group participation through social norms in 2,190 online science discussions. Proceedings of the National Academy of Sciences 116, 20 (2019), 9785–9789. find
  54. Peter O’Donovan, Aseem Agarwala, and Aaron Hertzmann. 2015. Designscape: Design with interactive layout suggestions. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 1221–1224. find
  55. Joon Sung Park, Joseph Seering, and Michael S. Bernstein. 2022. Measuring the Prevalence of Anti-Social Behavior in Online Communities. Proceedings of the ACM on Human-Computer Interaction CSCW (2022). find
  56. Gordon Pennycook, Adam Bear, Evan T Collins, and David G Rand. 2020. The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management Science 66, 11 (2020), 4944–4957. find
  57. Nicholas Proferes, Naiyan Jones, Sarah Gilbert, Casey Fiesler, and Michael Zimmer. 2021. Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics. Social Media + Society 7, 2 (2021), 20563051211019004. https://doi.org/10.1177/20563051211019004 arXiv:https://doi.org/10.1… — [doi:10.1177/20563051211019004](https://doi.org/10.1177/20563051211019004) ↗
  58. Prolific. 2022. Prolific: Quickly find research participants you can trust. https://www.prolific.co/ ↗
  59. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. find
  60. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 (2021). find
  61. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Vos, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-Shot Text-to-Image Generation. Proceedings of the 38th International Conference on Machine Learning (2021).
  62. Reddit. 2021. Reddit by the Numbers. https://www.redditinc.com/ ↗
  63. Marc Rettig. 1994. Prototyping for tiny fingers. Commun. ACM 37, 4 (1994), 21–27. find
  64. Horst WJ Rittel and Melvin M Webber. 1973. Dilemmas in a general theory of planning. Policy sciences 4, 2 (1973), 155–169. find
  65. David Rolf. 2015. The fight for $15: The right wage for a working America. New Press, The. find
  66. Matthew J. Salganik, Peter Sheridan Dodds, and Duncan J. Watts. 2006. Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market. Science 311, 5762 (2006), 854–856. https://doi.org/10.1126/science.1121066 arXiv:https://www.science.org/doi/pdf/10.1126/science.1121066 — [doi:10.1126/science.1121066](https://doi.org/10.1126/science.1121066) ↗
  67. Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: embedded single-camera sensing of printed physical user interfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 447–456. find
  68. Timo Schick and Hinrich Schütze. 2020. Few-shot text generation with pattern-exploiting training. arXiv preprint arXiv:2012.11926 (2020). — [arXiv:2012.11926](https://arxiv.org/abs/2012.11926) find
  69. D.A. Schön. 1984. The Reflective Practitioner: How Professionals Think In Action. Basic Books, Vol. 5126. Basic Books. find
  70. Reinhard Sefelin, Manfred Tscheligi, and Verena Giller. 2003. Paper prototyping-what is it good for? A comparison of paper-and computer-based low-fidelity prototyping. In CHI’03 extended abstracts on Human factors in computing systems. 778–779. find
  71. Carolyn Snyder. 2003. Paper prototyping: The fast and easy way to design and refine user interfaces. Morgan Kaufmann. find
  72. Sara Owsley Sood, Elizabeth F. Churchill, and Judd Antin. 2012. Automatic Identification of Personal Insults on Social News Sites. J. Am. Soc. Inf. Sci. Technol. 63, 2 (feb 2012), 270–285. https://doi.org/10.1002/asi.21690 — [doi:10.1002/asi.21690](https://doi.org/10.1002/asi.21690) ↗
  73. Don Tapscott and Art Caston. 1992. Paradigm shift: The new promise of information technology. McGraw-Hill, Inc. find
  74. Nathan TeBlunthuis, Aaron Shaw, and Benjamin Mako Hill. 2018. Revisiting" The rise and decline" in a population of peer production projects. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–7. find
  75. David Thomas. 2006. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 27 (2006). find
  76. Johan Ugander, Brian Karrer, Lars Backstrom, and Jon Kleinberg. 2013. Graph cluster randomization: Network exposure to multiple universes. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 329–337. find
  77. Emily A Vogels. 2021. The state of online harassment. Pew Research Center 13 (2021). find
  78. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems 32 (2019).
  79. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from Language Models. arXiv preprint arXiv:2112.04359 (2021).
  80. Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. PromptChainer: Chaining Large Language Model Prompts through Visual Programming. arXiv preprint arXiv:2203.06566 (2022). — [arXiv:2203.06566](https://arxiv.org/abs/2203.06566) find
  81. Tongshuang Wu, Michael Terry, and Carrie J Cai. 2021. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. arXiv preprint arXiv:2110.01691 (2021). — [arXiv:2110.01691](https://arxiv.org/abs/2110.01691) find
  82. Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 1433–1444. find
  83. Ya Xu, Nanyu Chen, Addrian Fernandez, Omar Sinno, and Anmol Bhasin. 2015. From infrastructure to culture: A/B testing challenges in large scale social networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2227–2236. find
  84. Indi Young. 2016. Describing Personas. Retrieved July 19, 2022 from https://medium.com/inclusive-software/describing-personas-af992e3fc527 ↗
  85. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems 32 (2019).
  86. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning. PMLR, PMLR, 12697–12706.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2022-social-simulacra-creating-populated-prototypes/