Elamur Research⌕
☰
home / papers / 2023 / Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties

Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties

Paper · 2023 · Taylor Sorensen, Liwei Jiang, Jena Hwang et al.

Summary

AI systems trained on aggregated labels tend to average away real disagreements between legitimate human values, such as honesty versus loyalty to a friend. This paper tests how well language models can represent such plural considerations explicitly. The authors used GPT-4, through symbolic knowledge distillation, to build ValuePrism, a corpus of contextualized values, rights, and duties attached to real situations written by people. They checked its quality with crowdworkers from varied demographic backgrounds. On this data they trained Value Kaleidoscope (Kaleido), an open multi-task model with four tasks:

  • generating relevant considerations,
  • judging whether a consideration is relevant,
  • classifying valence (supports, opposes, or depends on context),
  • explaining how a consideration bears on an action.

The KaleidoSys pipeline overgenerates candidates, filters them by relevance, and removes near-duplicates. Annotators judged its output more accurate and more complete than GPT-4's, even though GPT-4 was the teacher. Kaleido also helps explain why people disagree on ambiguous decisions and transfers to other ethical frameworks. The work offers an interpretable, modular basis for value-aware AI.

Key points

  • ValuePrism: about 218k values, rights, and duties linked to roughly 31k human-written situations. Annotators rated the GPT-4 outputs high-quality in 91% of cases and found missing considerations in under 1% of situations.
  • Model release: Kaleido comes in 5 sizes (e.g., allenai/kaleido-xl) on Hugging Face, with code and a demo. Access is restricted to research use on request.
  • KaleidoSys workflow: generates 100 candidates per situation, keeps those Kaleido rates as relevant, deduplicates by text similarity, and then scores relevance and valence.
  • Coverage of human rights: ValuePrism contains matches for 97.5% of the 41 rights the authors extracted from the UN Universal Declaration of Human Rights, retrieved via WANLI entailment and checked by hand.
  • Sensitivity and false balance: Kaleido shifts its relevance and valence scores when small details of a situation change. In a test on 20 hand-written clearly good or clearly bad actions, it produced no contrived supporting or opposing values.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓University of WashingtonSeattle · United States353 works · 200,043 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations🎓University of OxfordOxford · United Kingdom104 works · 157,183 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Artificial intelligence · 49 works

Cite

Cite as Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas and Yejin Choi. 2023. Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties. arXiv:2309.00779 · Vol. 38 No. 18: AAAI-24 Technical Tracks 18; 2024; 19937-19947.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1609/aaai.v38i18.29970 · arXiv 2309.00779

@misc{sorensen2023value,
  title = {{Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties}},
  author = {Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas and Yejin Choi},
  year = {2023},
  doi = {10.1609/aaai.v38i18.29970},
  eprint = {2309.00779},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1609/aaai.v38i18.29970}
}

References

86 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. find
  2. Chung, H. W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, Y.; Wang, X.; Dehghani, M.; Brahma, S.; Webson, A.; Gu, S. S.; Dai, Z.; Suzgun, M.; Chen, X.; Chowdhery, A.; Castro-Ros, A.; Pellat, M.; Robinson, K.; Valter, D.; Narang, S.; Mishra, G.; Yu, A.; Zhao, V.; Huang, Y.; Dai, A.; Yu, H… find
  3. Dancy, J. 2004. Ethics Without Principles. New York: Oxford University Press. Davani, A. M.; Dı́az, M.; and Prabhakaran, V. 2022. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. Transactions of the Association for Computational Linguistics, 10: 92–110.
  4. Driver, J. 2022. The History of Utilitarianism. In Zalta, E. N.; and Nodelman, U., eds., The Stanford Encyclopedia of Philosophy. find
  5. Metaphysics Research Lab, Stanford University, Winter 2022 edition. find
  6. Durmus, E.; Nyugen, K.; Liao, T. I.; Schiefer, N.; Askell, A.; Bakhtin, A.; Chen, C.; Hatfield-Dodds, Z.; Hernandez, D.; Joseph, N.; et al. 2023. Towards Measuring the Representation of Subjective Global Opinions in Language Models. arXiv preprint arXiv:2306.16388.
  7. Feng, S.; Park, C. Y.; Liu, Y.; and Tsvetkov, Y. 2023. From Pre-training Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models. arXiv:2305.08283. — [arXiv:2305.08283](https://arxiv.org/abs/2305.08283) find
  8. Festinger, L. 1962. Cognitive dissonance. Sci. Am., 207(4): 93– 102. find
  9. Forbes, M.; Hwang, J. D.; Shwartz, V.; Sap, M.; and Choi, Y. 2021. Social Chemistry 101: Learning to Reason about Social and Moral Norms. arXiv:2011.00620. Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J. W.; Wallach, H. M.; Daumé, H.; and Crawford, K. 2018. Datasheets for datasets. Communica… find
  10. Gilardi, F.; Alizadeh, M.; and Kubli, M. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30).
  11. Gill, M. B.; and Nichols, S. 2008. Sentimentalist Pluralism: Moral Psychology and Philosophical Ethics. Philosophical Issues, 18(1): 143–163. find
  12. Gordon, M. L.; Lam, M. S.; Park, J. S.; Patel, K.; Hancock, J.; Hashimoto, T.; and Bernstein, M. S. 2022. Jury Learning: Integrating Dissenting Voices into Machine Learning Models. In CHI Conference on Human Factors in Computing Systems. find
  13. ACM. Gowans, C. 2021. Moral Relativism. In Zalta, E. N., ed., The Stanford Encyclopedia of Philosophy. find
  14. Metaphysics Research Lab, Stanford University, Spring 2021 edition. find
  15. Griffiths, N. 2021. Personal Values & Decision-Making Biases. In Personal Values & Decision-Making Biases. find
  16. Hendrycks, D.; Burns, C.; Basart, S.; Critch, A.; Li, J.; Song, D.; and Steinhardt, J. 2023. Aligning AI With Shared Human Values. arXiv:2008.02275.
  17. Hsieh, C.-Y.; Li, C.-L.; Yeh, C.-k.; Nakhost, H.; Fujii, Y.; Ratner, A.; Krishna, R.; Lee, C.-Y.; and Pfister, T. 2023. Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes. In Rogers, A.; Boyd-Graber, J.; and Alexander, L.; and Moore, M. 2021…
  18. Metaphysics Research Lab, Stanford University, Winter 2021 edition. find
  19. Allen Insitute for AI. 2023. AI2 ImpACT Licenses. https://allenai.org/impact-license. [accessed August 2023]. ↗
  20. Archive, G. D. 2011. European Values Study Longitudinal Data File 1981-2008 (EVS 1981-2008). EVS. find
  21. Argyle, L. P.; Bail, C. A.; Busby, E. C.; Gubler, J. R.; Howe, T.; Rytting, C.; Sorensen, T.; and Wingate, D. 2023a. Leveraging AI for democratic discourse: Chat interventions can improve online political conversations at scale. Proceedings of the National Academy of Sciences, 120(41): e2311627120. find
  22. Argyle, L. P.; Busby, E. C.; Fulda, N.; Gubler, J. R.; Rytting, C.; and Wingate, D. 2023b. Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis, 1–15.
  23. Aroyo, L.; Taylor, A. S.; Diaz, M.; Homan, C. M.; Parrish, A.; Serapio-Garcia, G.; Prabhakaran, V.; and Wang, D. 2023. DICES Dataset: Diversity in Conversational AI Evaluation for Safety. arXiv:2306.11247. — [arXiv:2306.11247](https://arxiv.org/abs/2306.11247) find
  24. Baan, J.; Aziz, W.; Plank, B.; and Fernández, R. 2022. Stop Measuring Calibration When Humans Disagree. arXiv:2210.16133. find
  25. Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; Chen, C.; Olsson, C.; Olah, C.; Hernandez, D.; Drain, D.; Ganguli, D.; Li, D.; Tran-Johnson, E.; Perez, E.; Kerr, J.; Mueller, J.; Ladish, J.; Landau, J.; Ndousse, K.; Lukosuite,… — Constitutional AI: Harmlessness from AI Feedback
  26. Bhagavatula, C.; Hwang, J. D.; Downey, D.; Bras, R. L.; Lu, X.; Qin, L.; Sakaguchi, K.; Swayamdipta, S.; West, P.; and Choi, Y. 2023. I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation. arXiv:2212.09246. find
  27. Boykoff, M. T.; and Boykoff, J. M. 2004. Balance as bias: global warming and the US prestige press. Global Environmental Change, 14(2): 125–136. find
  28. Britannica Editors. 2002. Pluralism — Ideology, Diversity & Tolerance — britannica.com. https://www.britannica.com/topic/ pluralism-politics. [Accessed 07-08-2023]. ↗
  29. Brosch, T.; and Sander, D. 2013. Neurocognitive mechanisms underlying value-based decision-making: from core values to economic value. Frontiers in Human Neuroscience, 7. find
  30. Casella, G.; and George, E. I. 1992. Explaining the Gibbs Sampler. The American Statistician, 46(3): 167–174. find
  31. Casper, S.; Davies, X.; Shi, C.; Gilbert, T. K.; Scheurer, J.; Rando, J.; Freedman, R.; Korbak, T.; Lindner, D.; Freire, P.; Wang, T.; Marks, S.; Segerie, C.-R.; Carroll, M.; Peng, A.; Christoffersen, P.; Damani, M.; Slocum, S.; Anwar, U.; Siththaranjan, A.; Nadeau, M.; Michaud, E. J.; Pfau, J.; Kra… find
  32. Chang, R. 1997. Incommensurability, Incomparability, and Practical Reason. Cambridge, MA, USA: Harvard. find
  33. Okazaki, N., eds., Findings of the Association for Computational Linguistics: ACL 2023, 8003–8017. Toronto, Canada: Association for Computational Linguistics. find
  34. Hwang, J. D.; Bhagavatula, C.; Bras, R. L.; Da, J.; Sakaguchi, K.; Bosselut, A.; and Choi, Y. 2021. find
  35. COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs. arXiv:2010.05953.
  36. Imundo, M.; and Rapp, D. 2021. When Fairness is Flawed: Effects of False Balance Reporting and Weight-of-Evidence Statements on Beliefs and Perceptions of Climate Change. Journal of Applied Research in Memory and Cognition, 11. find
  37. Jiang, L.; Hwang, J. D.; Bhagavatula, C.; Bras, R. L.; Liang, J.; Dodge, J.; Sakaguchi, K.; Forbes, M.; Borchardt, J.; Gabriel, S.; Tsvetkov, Y.; Etzioni, O.; Sap, M.; Rini, R.; and Choi, Y. 2022. Can Machines Learn Morality? The Delphi Experiment. arXiv:2110.07574. Johnson, M.; Schuster, M.; Le, Q.… find
  38. Jung, J.; West, P.; Jiang, L.; Brahman, F.; Lu, X.; Fisher, J.; Sorensen, T.; and Choi, Y. 2023. Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing. arXiv:2305.16635. Kant, I. 1785/2002. Groundwork for the Metaphysics of Morals.
  39. Yale University Press. Keeney, R. L. 1992. Value-Focused Thinking: A Path to Creative Decisionmaking. Harvard University Press. ISBN 9780674931978. find
  40. Kekes, J. 1993. The Morality of Pluralism. find
  41. Princeton University Press. Kim, H.; Hessel, J.; Jiang, L.; West, P.; Lu, X.; Yu, Y.; Zhou, P.; Bras, R. L.; Alikhani, M.; Kim, G.; Sap, M.; and Choi, Y. 2023. SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization. arXiv:2212.10465.
  42. Komppula, R.; Honkanen, A.; Rossi, S.; Kolesnikova, N.; et al. 2018. The impact of values on sustainable behaviour-A study among Russian and Finnish university students. European Journal of Tourism Research, 19: 116–131. find
  43. Landemore, H. 2013. Deliberation, cognitive diversity, and democratic inclusiveness: an epistemic argument for the random selection of representatives. Synthese, 190: 1209–1231. find
  44. Larmore, C. E. 1987. Patterns of Moral Complexity. find
  45. New York: Cambridge University Press. Lin, C.-Y. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out, 74–81. Barcelona, Spain: Association for Computational Linguistics.
  46. Liscio, E.; Araque, O.; Gatti, L.; Constantinescu, I.; Jonker, C.; Kalimeri, K.; and Murukannaiah, P. K. 2023. What does a Text Classifier Learn about Morality? An Explainable Method for Cross-Domain Comparison of Moral Rhetoric. In Proceedings of the 61st Annual Meeting of the Association for Compu… find
  47. Litman, L.; Robinson, J.; and Abberbock, T. 2017. TurkPrime. com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior research methods, 49(2): 433–442. find
  48. Liu, A.; Swayamdipta, S.; Smith, N. A.; and Choi, Y. 2022. WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation. arXiv:2201.05955.
  49. Lu, X. 2023. Learning Ambiguity from Crowd Sequential Annotations. arXiv:2301.01579. — [arXiv:2301.01579](https://arxiv.org/abs/2301.01579) find
  50. Lu, X.; Brahman, F.; West, P.; Jang, J.; Chandu, K.; Ravichander, A.; Qin, L.; Ammanabrolu, P.; Jiang, L.; Ramnath, S.; et al. 2023. Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning. arXiv preprint arXiv:2305.15065. find
  51. Lu, X.; Welleck, S.; Hessel, J.; Jiang, L.; Qin, L.; West, P.; Ammanabrolu, P.; and Choi, Y. 2022. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35: 27591–27609.
  52. Martı́, J. L. 2017. Pluralism and consensus in deliberative democracy. Critical Review of International Social and Political Philosophy, 20(5): 556–579. find
  53. Mason, E. 2006. Value pluralism. find
  54. Nguyen, K.; Daumé III, H.; and Boyd-Graber, J. 2017. Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 1464–1474. find
  55. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774.
  56. Plank, B. 2022. The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 10671–10682. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics.
  57. Prabhakaran, V.; Mitchell, M.; Gebru, T.; and Gabriel, I. 2022. A Human Rights-Based Approach to Responsible AI. arXiv:2210.02667. find
  58. Pyatkin, V.; Hwang, J. D.; Srikumar, V.; Lu, X.; Jiang, L.; Choi, Y.; and Bhagavatula, C. 2023. ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vo… find
  59. Páez, J.; De-Juanas, A.; Garcı́a-Castilla, F.; and Muelas, A. 2020. Relationship Between Basic Human Values and Decision-Making Styles in Adolescents. International Journal of Environmental Research and Public Health, 17(22): 8315. [Accessed 31-07-2023]. find
  60. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv:1910.10683.
  61. Ramamurthy, R.; Ammanabrolu, P.; Brantley, K.; Hessel, J.; Sifa, R.; Bauckhage, C.; Hajishirzi, H.; and Choi, Y. 2022. Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization. In The Eleventh International C…
  62. Randolph, J. J. 2005. Free-Marginal Multirater Kappa (multirater K [free]): An Alternative to Fleiss’ Fixed-Marginal Multirater Kappa. Online submission. find
  63. Rytting, C. M.; Sorensen, T.; Argyle, L.; Busby, E.; Fulda, N.; Gubler, J.; and Wingate, D. 2023. Towards Coding Social Science Datasets with Language Models. arXiv:2306.02177. — [arXiv:2306.02177](https://arxiv.org/abs/2306.02177) find
  64. Santurkar, S.; Durmus, E.; Ladhak, F.; Lee, C.; Liang, P.; and Hashimoto, T. 2023. Whose Opinions Do Language Models Reflect? arXiv:2303.17548.
  65. Santy, S.; Liang, J. T.; Bras, R. L.; Reinecke, K.; and Sap, M. 2023. NLPositionality: Characterizing Design Biases of Datasets and Models. arXiv:2306.01943.
  66. Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; and Smith, N. A. 2019. The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 1668–1678. Florence, Italy: Association for Computational Linguistics.
  67. Sap, M.; Gabriel, S.; Qin, L.; Jurafsky, D.; Smith, N. A.; and Choi, Y. 2020. Social Bias Frames: Reasoning about Social and Power Implications of Language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5477–5490. Online: Association for Computational Li…
  68. Scherrer, N.; Shi, C.; Feder, A.; and Blei, D. M. 2023. Evaluating the Moral Beliefs Encoded in LLMs. arXiv:2307.14324. find
  69. Sclar, M.; West, P.; Kumar, S.; Tsvetkov, Y.; and Choi, Y. 2022. Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 9649–9668. Abu Dhabi, Unite… find
  70. Stocker, M. 1990. Plural and Conflicting Values. find
  71. New York: Oxford University Press. Talat, Z.; Blix, H.; Valvoda, J.; Ganesh, M. I.; Cotterell, R.; and Williams, A. 2022. On the Machine Learning of Ethical Judgments from Natural Language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lingu… find
  72. Tasioulas, J. 2022. Artificial Intelligence, Humanistic Ethics. Daedalus, 151(2): 232–243. find
  73. Wallach, W.; and Allen, C. 2008. Moral machines: Teaching robots right from wrong. find
  74. Oxford University Press. Wang, P.; Wang, Z.; Li, Z.; Gao, Y.; Yin, B.; and Ren, X. 2023a. SCOTT: Self-Consistent Chain-of-Thought Distillation. arXiv:2305.01879.
  75. Wang, Y.; Ivison, H.; Dasigi, P.; Hessel, J.; Khot, T.; Chandu, K. R.; Wadden, D.; MacMillan, K.; Smith, N. A.; Beltagy, I.; and Hajishirzi, H. 2023b. How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources. arXiv:2306.04751.
  76. Wenar, L. 2023. Rights. In Zalta, E. N.; and Nodelman, U., eds., The Stanford Encyclopedia of Philosophy. find
  77. Metaphysics Research Lab, Stanford University, Spring 2023 edition. find
  78. West, P.; Bhagavatula, C.; Hessel, J.; Hwang, J.; Jiang, L.; Le Bras, R.; Lu, X.; Welleck, S.; and Choi, Y. 2022a. Symbolic Knowledge Distillation: from General Language Models to Commonsense Models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computati…
  79. West, P.; Bhagavatula, C.; Hessel, J.; Hwang, J. D.; Jiang, L.; Bras, R. L.; Lu, X.; Welleck, S.; and Choi, Y. 2022b. Symbolic Knowledge Distillation: from General Language Models to Commonsense Models. arXiv:2110.07178.
  80. West, P.; Bras, R. L.; Sorensen, T.; Lin, B. Y.; Jiang, L.; Lu, X.; Chandu, K.; Hessel, J.; Baheti, A.; Bhagavatula, C.; and Choi, Y. 2023. NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation. arXiv:2312.05979. find
  81. Williams, B. 1985. Moral Luck. Critica, 17(51): 101–105. find
  82. Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; Davison, J.; Shleifer, S.; von Platen, P.; Ma, C.; Jernite, Y.; Plu, J.; Xu, C.; Scao, T. L.; Gugger, S.; Drame, M.; Lhoest, Q.; and Rush, A. M. 2020. HuggingFace’s Transformers: State…
  83. Wu, Z.; Hu, Y.; Shi, W.; Dziri, N.; Suhr, A.; Ammanabrolu, P.; Smith, N. A.; Ostendorf, M.; and Hajishirzi, H. 2023. Fine-Grained Human Feedback Gives Better Rewards for Language Model Training. arXiv preprint arXiv:2306.01693.
  84. Xu, C.; Guo, D.; Duan, N.; and McAuley, J. 2023. Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data. arXiv:2304.01196.
  85. Zhou, X.; Zhu, H.; Yerukola, A.; Davidson, T.; Hwang, J. D.; Swayamdipta, S.; and Sap, M. 2023. COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements. In Findings of ACL. find
  86. Ziems, C.; Held, W.; Shaikh, O.; Chen, J.; Zhang, Z.; and Yang, D. 2023. Can Large Language Models Transform Computational Social Science? arXiv:2305.03514.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sorensen-2023-value-kaleidoscope-engaging-ai-pluralistic/