Elamur Research⌕
☰
home / papers / 2026 / Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts

Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts

Paper · 2026 · Yi Ren, Lewis Mitchell and Matthew Roughan

Summary

Moral classifiers for social media usually merge several crowd annotators' labels into one target. This hides real disagreement, which comes from differing moral outlooks and from short, ambiguous tweets. The authors take a perspectivist approach and predict what each individual annotator would say. They use the Moral Foundations Twitter Corpus (MFTC), which has sparse and uneven annotator coverage, and fine-tune BERT with an added Annotator Layer. This component learns how each annotator's judgements differ from the shared text representation. It comes in a linear-transformation form (LT-AL) and a bias-only form. Predicting individual annotations works clearly better than with plain fine-tuned BERT. The learned annotator parameters also reveal interpretable tendencies toward particular foundations. The gains are largest for the binding foundations, where annotators agree least. The study suggests that disagreement is structured signal rather than noise, and that evaluating on aggregated labels can make models look better than they are. One weakness remains: producing a good single consensus label is still hard, especially with the LT variant.

Key points

  • Data: MFTC (Hoover et al., 2020) contains 35,108 tweets on seven hashtag-defined topics, labelled by 23 trained annotators with 3–8 labels per tweet. The moral classes are a minority (about 9–20% of annotations per foundation), and the co-annotation network is fragmented.
  • Annotator workload: workload is very uneven. Each annotator labelled 5,585 tweets on average (median 4,588, range 560–19,556, s.d. 4,747).
  • Accuracy: compared with fine-tuned BERT, the annotator-aware model improves classification accuracy by 10.2% on average across the five foundations.
  • Gains by foundation: mean F1 gains of LT-AL over FT-BERT are larger for the binding foundations, Authority (19.9%) and Purity (19.3%). They are smaller for the individualising foundations, Care (10.4%) and Fairness (6.8%). Loyalty sits in between at 11.4%. This pattern matches the lower Krippendorff's α observed for the binding foundations.
  • Limitations: annotator demographic and MFQ metadata were not available to the authors. In addition, averaging over all annotator heads to get aggregated predictions adds noise. The bias-only variant is roughly on par with FT-BERT on aggregated labels, while LT-AL drops markedly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of AdelaideAdelaide · Australia6 works · 1,417 citations🎓Adelaide UniversityAdelaide · Australia5 works · 3 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

Cite

Cite as Yi Ren, Lewis Mitchell and Matthew Roughan. 2026. Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts. Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.nlpcss-1.4

@inproceedings{ren2026learning,
  title = {{Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts}},
  author = {Yi Ren, Lewis Mitchell and Matthew Roughan},
  booktitle = {Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science},
  year = {2026},
  doi = {10.18653/v1/2026.nlpcss-1.4},
  url = {https://doi.org/10.18653/v1/2026.nlpcss-1.4}
}

References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anny D. Alvarez Nogales and Oscar Araque. 2024. Moral disagreement over serious matters: Discovering the knowledge hidden in the perspectives. In Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024, pages 67–77, Torino, Italia. ELRA and ICCL. find
  2. Jesse Graham and Jonathan Haidt. 2012. The moral foundations dictionary. https://moralfoundations.org/other-materials/. Accessed: 25 November 2025. ↗
  3. Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in Experimental Social Psychology, Advances in experimental social psychology, pages 55–130. Elsevier. find
  4. Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. 2020. MoralStrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction. Knowl. Based Syst., 191(105184):105184. find
  5. Mohammad Atari, Jonathan Haidt, Jesse Graham, Sena Koleva, Sean T Stevens, and Morteza Dehghani. 2023. Morality beyond the WEIRD: How the nomological network of morality varies across cultures. J. Pers. Soc. Psychol., 125(5):1157–1188. find
  6. Jesse Graham, Jonathan Haidt, and Brian A Nosek. 2009. Liberals and conservatives rely on different sets of moral foundations. J. Pers. Soc. Psychol., 96(5):1029–1046. find
  7. Mariano Gastón Beiró, Jacopo D’Ignazi, Victoria Perez Bustos, María Florencia Prado, and Kyriaki Kalimeri. 2023. Moral narratives around the vaccination debate on facebook. In Proceedings of the ACM Jesse Graham, Brian A Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H Ditto. 2011. Map… find
  8. Siyi Guo, Negar Mokhberian, and Kristina Lerman. 2023. A data fusion framework for multi-domain morality learning. Proceedings of the International AAAI Conference on Web and Social Media, 17:281– 291. find
  9. Klaus Krippendorff. 2018. Content Analysis: An Introduction to Its Methodology, 4th edition. Sage Publications, Thousand Oaks, CA. find
  10. Ying Lin, Joe Hoover, Gwenyth Portillo-Wightman, Christina Park, Morteza Dehghani, and Heng Ji. 2018. Acquiring background knowledge to improve moral value prediction. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 552–559. find
  11. J Haidt, S H Koller, and M G Dias. 1993. Affect, culture, and morality, or is it wrong to eat your dog? J. Pers. Soc. Psychol., 65(4):613–628. find
  12. Jonathan Haidt. 2001. The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychol. Rev., 108(4):814–834. find
  13. Enrico Liscio, Alin E. Dondera, Andrei Geadău, Catholijn M. Jonker, and Pradeep K. Murukannaiah. 2022. Cross-domain classification of moral values. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2727–2745, Seattle, United States. Association for Computational Lingui… find
  14. Jonathan Haidt. 2012. The Righteous Mind: Why Good People are Divided by Politics and Religion. Penguin UK. find
  15. Jonathan Haidt and Jesse Graham. 2007. When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize. Soc. Justice Res., 20(1):98–116. find
  16. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  17. Joe Hoover, Gwenyth Portillo-Wightman, Leigh Yeh, Shreya Havaldar, Aida Mostafazadeh Davani, Ying Lin, Brendan Kennedy, Mohammad Atari, Zahra Kamel, Madelyn Mendlen, Gabriela Moreno, Christina Park, Tingyee E Chang, Jenna Chin, Christian Leong, Jun Yen Leung, Arineh Mirinjian, and Morteza Dehghani.… find
  18. Negar Mokhberian, Myrl Marmarelis, Frederic Hopp, Valerio Basile, Fred Morstatter, and Kristina Lerman. 2024. Capturing perspectives of crowdsourced annotators in subjective learning tasks. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingu… find
  19. Frederic R Hopp, Jacob T Fisher, Devin Cornell, Richard Huskey, and René Weber. 2021. The extended moral foundations dictionary (eMFD): Development and applications of a crowd-sourced approach to extracting moral intuitions from text. Behav. Res. Methods, 53(1):232–246. find
  20. Tuan Dung Nguyen, Ziyu Chen, Nicholas George Carroll, Alasdair Tran, Colin Klein, and Lexing Xie. 2024. Measuring moral dimensions in social media with mformer. Proceedings of the International AAAI Conference on Web and Social Media, 18:1134– 1147. find
  21. Xiaolei Huang, Alexandra Wormley, and Adam Cohen. 2022. Learning to adapt domain shifts of moral values via instance weighting. In Proceedings of the 33rd ACM Conference on Hypertext and Social Media, New York, NY, USA. ACM. find
  22. Matthias Orlikowski, Jiaxin Pei, Paul Röttger, Philipp Cimiano, David Jurgens, and Dirk Hovy. 2025. Beyond demographics: Fine-tuning large language models to predict individuals’ subjective text perceptions. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (…
  23. Julie Jiang, Luca Luceri, and Emilio Ferrara. 2025. Moral values underpinning COVID-19 online communication patterns. In Companion Proceedings of the ACM on Web Conference 2025, pages 2642–2650, New York, NY, USA. ACM. find
  24. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 others. 20… — PyTorch: An Imperative Style, High-Performance Deep Learning Library
  25. Kamil Kanclerz, Marcin Gruza, Konrad Karanowski, Julita Bielaniewicz, Piotr Milkowski, Jan Kocon, and Przemyslaw Kazienko. 2022. What if ground truth is subjective? personalized deep neural hate speech detection. In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, pages… find
  26. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. find
  27. Scikit-learn: Spassena P Koleva, Jesse Graham, Ravi Iyer, Peter H Ditto, and Jonathan Haidt. 2012. Tracing the threads: How five moral concerns (especially Purity) help explain culture war attitudes. J. Res. Pers., 46(2):184– 194. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ren-2026-learning-moral-diversity-modelling-individual/