Elamur Research⌕
☰
home / papers / 2026 / Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes

Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes

Summary

LLMs are increasingly proposed for spotting violent content online, but little is known about how they reason through ambiguous everyday conflicts or whether that reasoning shifts with the identity of the person involved. The authors adapt the Violent Behavior Vignette Questionnaire (VBVQ), an established psychology tool built from interviews about real conflicts, and present its ten provocation scenarios to six instruction-tuned LLMs in a zero-shot setting. Each prompt asks the model to respond as a particular persona defined by race, age, and US county. Models answer in open-ended and multiple-choice formats. The authors examine both the generated text and the likelihoods the model assigns to response sequences. They find that what models write often differs from what their internal probabilities favor. Their implied violence also varies across demographics in ways that conflict with criminological evidence. The study shows that alignment checks based only on visible output can miss hidden biases, and the authors advise against using current LLMs in community violence intervention work.

Key points

  • Models tested: Mistral-7B-Instruct-v0.1, Mixtral-8x22B-Instruct-v0.1, DeepSeek-Distill Qwen (including a 32B variant), GPT-4o-mini, and Llama3.1-8B-Instruct. Llama3.1-8B-Instruct replaced GPT-4o-mini for the likelihood analysis because the GPT-4o-mini API did not provide token log-probabilities.
  • Prompt design: 6 racial/ethnic groups × 6 US counties × 3 ages (15, 25, 35) across 10 vignettes, giving 1,080 persona prompts per model. Five samples were drawn per prompt at temperature 0.7 using self-consistency decoding.
  • Response formats and references: answers were scored against the original 10 VBVQ options and a three-way grouping (Pacifist / Non-Violent / Violent). GPT-4o generated four paraphrases of each human-labeled reference answer. For comparison, earlier human studies found violent choices about 25% of the time.
  • Age finding: every model rated 15-year-old personas as less violent than older ones, which reverses the pattern reported in criminology. One exception: with human-written labels alone, the Qwen models showed the expected age direction.
  • Location and race findings: location effects were inconsistent across models; for example, Denver appeared more violent for Qwen but less so for Llama and Mixtral. Racial results suggested politically motivated overcorrection, which was weaker in larger models. The evaluation framework and expert-annotated data are slated for release.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3b7ht2jn59d3

@inproceedings{myers2026uncovering,
  title = {{Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes}},
  author = {Myers, Quintin and Gao, Yanjun},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3b7ht2jn59d3},
  url = {https://doi.org/10.63317/3b7ht2jn59d3}
}

References

44 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michelle Alexander. 2012. The New Jim Crow: Mass Incarceration in the Age of Colorblindness, revised edition edition. The New Press, New York. find
  2. Ayman Alhelbawy, Poesio Massimo, and Udo Kruschwitz. 2016. Towards a corpus of violence acts in Arabic social media. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC‘16), pages 1627– 1631, Portorož, Slovenia. European Language Resources Association (ELR… find
  3. Pierpaolo Basile, Marco Degemmis, Marco Polignano, Giovanni Semeraro, Lucia Siciliani, Vincenzo Tamburrano, Fabiana Battista, and Rosa Scardigno. 2024. Llamantino against cyber intimate partner violence. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages… find
  4. CDC. 2025. About Community Violence. find
  5. Rosalind S. Chou. 2015. Asian American Sexual Politics: The Construction of Race, Gender, and Sexuality, reprint edition edition. Rowman & Littlefield Publishers, Lanham, Maryland. find
  6. Rosalind S. Chou and Joe R. Feagin. 2014. Myth of the Model Minority: Asian Americans Facing Racism, Second Edition, 2 edition edition. Routledge, Boulder. find
  7. Joe R. Feagin. 1991. The Continuing Significance of Race: Antiblack Discrimination in Public Places. American Sociological Review, 56(1):101–116. find
  8. Federal Bureau of Investigation. 2025. Crime Data Explorer (CDE). https://cde.ucr.cjis.gov/LATEST/webapp/\#/pages/ Accessed: explorer/crime/crime-trend. 2025-05-16. ↗
  9. William R. Frey, Desmond U. Patton, Michael B. Gaskell, and Kyle A. McGregor. 2020. Artificial Intelligence and Inclusion: Formerly Gang-Involved Youth as Domain Experts for Analyzing Unstructured Twitter Data. Social Science Computer Review, 38(1):42–56. Publisher: SAGE Publications Inc. find
  10. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097–1179. find our card
  11. Kimberly Glasgow and Ronald Schouten. 2014. Assessing violence risk in threatening communications. find
  12. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. find our card
  13. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654. find our card
  14. Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, and Jon Roozenbeek. 2025. Generative language models exhibit social identity biases. Nature Computational Science, 5(1):65–75. find
  15. Yibo Hu, MohammadSaleh Hosseini, Erick Skorupa Parolin, Javier Osorio, Latifur Khan, Patrick Brandt, and Vito D’Orazio. 2022. ConfliBERT: A pre-trained language model for political conflict and violence. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Compu… find
  16. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. find our card
  17. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  18. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. find
  19. Minju Kim, Heuiyeen Yeen, and Myoung-Wan Koo. 2024. Towards context-based violence detection: A korean crime dialogue dataset. In Findings of the Association for Computational Linguistics: EACL 2024, pages 603–623. find
  20. Shana Kleiner, Jessica A. Grieser, Shug Miller, James Shepard, Javier Garcia-Perez, Nick Deas, Desmond U. Patton, Elsbeth Turcan, and Kathleen McKeown. 2024. Unmasking camouflage: exploring the challenges of large language models in deciphering African American language & online performativity. AI a… find
  21. Jeffrey Lane. 2018. The Digital Street. Oxford University Press, New York. find
  22. Matthew T. Lee and Ramiro Martinez. 2009. Immigration reduces crime: an emerging scholarly consensus. In William F. Mcdonald, editor, Sociology of Crime, Law and Deviance, volume 13, pages 3–16. Emerald Group Publishing Limited. find
  23. Ramiro Martinez, Jacob I. Stowell, and Matthew T. Lee. 2010. Immigration and Crime in an Era of Transformation: A Longitudinal Analysis of Homicides in San Diego Neighborhoods, 1980–2000*. Criminology, 48(3):797–829. find
  24. Patrick E McKight and Julius Najab. 2010. Kruskal-wallis test. The corsini encyclopedia of psychology, pages 1–1. find
  25. Kevin L. Nunes, Chantal A. Hermann, Sacha Maimone, Maya Atlas, and Brian A. Grant. 2021. The Violent Behavior Vignette Questionnaire (VBVQ): A Measure of Violent Behavior for Research in Forensic and Non-Forensic Settings and Populations. Psychological Reports, 124(4):1863–1896. Publisher: SAGE Publ… find
  26. Kevin L. Nunes, Chloe I. Pedneault, Chantal A. Hermann, and Julia M. Fraser. 2023. Do violent attitude measures assess evaluative attitudes toward violence? Psychology of Violence, 13(4):348–359. Publisher: Educational Publishing Foundation. find
  27. Desmond U. Patton, William R. Frey, Kyle A. Mc-Gregor, Fei-Tzin Lee, Kathleen McKeown, and Emanuel Moss. 2020. Contextual Analysis of Social Media: The Promise and Challenge of Eliciting Context in Social Media Posts with Natural Language Processing. In Proceedings of the AAAI/ACM Conference on AI,… find
  28. Desmond Upton Patton, Robert D. Eschmann, and Dirk A. Butler. 2013. Internet banging: New trends in social media, gang violence, masculinity and hip hop. Computers in Human Behavior, 29(5):A54–A59. find
  29. Desmond Upton Patton, Robert D. Eschmann, Caitlin Elsaesser, and Eddie Bocanegra. 2016. Sticks, stones and Facebook accounts: What violence outreach workers know about social media and urban-based gang violence in Chicago. Computers in Human Behavior, 65:591–600. find
  30. Desmond Upton Patton, Patrick Leonard, Caitlin Elaesser, Robert D. Eschmann, Sadiq Patel, and Shantel Crosby. 2019. What’s a Threat on Social Media? How Black and Latino Chicago Young Men Define and Navigate Threats Online. Youth & Society, 51(6):756–772. Publisher: SAGE Publications Inc. find
  31. Desmond Upton Patton, Kyle McGregor, and Gary Slutkin. 2018. Youth Gun Violence Prevention in a Digital Age. Pediatrics, 141(4):e20172438. find
  32. Ellie Pavlick, Heng Ji, Xiaoman Pan, and Chris Callison-Burch. 2016. The gun violence database: A new task and data set for nlp. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1018–1024. find
  33. David C. Pyrooz, Scott H. Decker, and Richard K. Moule Jr. 2015. Criminal and Routine Activities in Online Settings: Gangs, Offenders, and the Internet. Justice Quarterly, 32(3):471–499. Publisher: Routledge _eprint: https://doi.org/10.1080/07418825.2013.778326. — [doi:10.1080/07418825.2013.778326](https://doi.org/10.1080/07418825.2013.778326) ↗
  34. Sourav Saha, Jahedul Alam Junaed, Maryam Saleki, Arnab Sen Sharma, Mohammad Rashidujjaman Rifat, Mohamed Rahouti, Syed Ishtiaque Ahmed, Nabeel Mohammed, and Mohammad Ruhul Amin. 2023. Vio-lens: A novel dataset of annotated social network posts leading to different forms of communal violence and its… find
  35. Robert M. Sapolsky. 2018. Behave: The Biology of Humans at Our Best and Worst, reprint edition edition. Penguin Books, New York, New York. find
  36. Mohammad Shokri, Emily Klapper, Jason Shan, and Sarah Ita Levitan. 2025. Finding common patterns in domestic violence stories posted on reddit. In Proceedings of the The 7th Workshop on Narrative Understanding, pages 15–25. find
  37. Elizabeth P. Shulman, Laurence D. Steinberg, and Alex R. Piquero. 2013. The Age–Crime Curve in Adolescence and Early Adulthood is Not Due to Age Differences in Economic Status. Journal of Youth and Adolescence, 42(6):848–860. find
  38. Darrell Steffensmeier, Jessie Slepicka, and Jennifer Schwartz. 2025. International and Historical Variation in the Age–Crime Curve. Annual Review of Criminology, 8(Volume 8, 2025):239–268. Publisher: Annual Reviews. find
  39. Lisa Stolzenberg and Stewart J. D’Alessio. 2008. Co-Offending and the Age-Crime Curve. Journal of Research in Crime and Delinquency, 45(1):65– 86. Publisher: SAGE Publications Inc. find
  40. Hetong Wang, Pasquale Minervini, and Edoardo Ponti. 2024. Probing the emergence of cross-lingual alignment during LLM training. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12159–12173, Bangkok, Thailand. Association for Computational Linguistics. find
  41. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations. find our card
  42. Magdalena Wolska, Matti Wiegmann, Christopher Schröder, Ole Borchardt, Benno Stein, and Martin Potthast. 2023. Trigger warnings: Bootstrapping a violence detector for fan fiction. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 569–576. find
  43. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find
  44. Caleb Ziems and Diyi Yang. 2021. To protect and to serve? analyzing entity-centric framing of police violence. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 957–976, Punta Cana, Dominican Republic. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/myers-2026-uncovering-hidden-violent-tendencies-llms/