Elamur Research⌕
☰
home / papers / 2026 / Taken into consideration: evaluating esteem biases by race, gender and region in large language models in Brazilian Portuguese

Taken into consideration: evaluating esteem biases by race, gender and region in large language models in Brazilian Portuguese

Original (Portuguese): Levados em Consideração: Uma Avaliação de Vieses de Estima por Raça, Gênero e Região em Grandes Modelos de Linguagem em Português Brasileiro

Paper · 2026 · João Lucas Lima de Melo and Marlo Souza

Summary

This PROPOR 2026 paper examines whether large language models show differences in esteem toward social groups in Brazilian Portuguese. Esteem here means the level of respect or deference a model shows toward a group. The authors also look at intersectional subjects, which earlier Portuguese bias studies rarely covered. They tested GPT-4o, GPT-4o-mini, Sabiá-3 and Sabiázinho-3.

The test material was built from 20 template sentences, half positive and half negative, each with a <SUJEITO> slot. The slot was filled with subjects carrying explicit markers of race/colour, binary gender and Brazilian macro-region, used alone or combined. Each model was prompted with and without a simple persona-based jailbreak.

Adding social markers generally lowered esteem compared with a neutral "pessoa/indivíduo" subject. Racial markers caused the largest drops. The jailbreak had mixed effects: depending on model and marker, it could widen or narrow the gaps.

The study adds region as a bias dimension specific to Brazil. It argues that Portuguese-language LLMs need ongoing auditing.

Key points

  • Social markers:
    • Race categories come from the IBGE 2022 Census, with "asiática" used instead of "amarela" because "amarela" scored systematically lower in pilot runs.
    • Gender is binary (homem/mulher).
    • Regions are the five Brazilian macro-regions.
    • Neutral references are "a pessoa" and "o indivíduo".
  • Data size: the authors report 24 single-marker subjects, which combine into 144 subjects with one to three markers. Across conditions this yields 20,160 esteem assignments.
  • Racial effects: in every model, the steepest esteem declines involved subjects marked "preta", "parda" and "asiática".
  • Sabiá-3: it had the most positive deviations from the neutral baseline (six, five of them race-neutral). Its largest single increase was +0.300, for the combined subject "homem" + "norte".
  • Jailbreak effects: the persona jailbreak sharply reduced esteem in GPT-4o-mini but raised it broadly in Sabiázinho-3, including for vulnerable groups. The authors conclude that it changes how responses are distributed rather than simply amplifying bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as João Lucas Lima de Melo and Marlo Souza. 2026. Levados em Consideração: Uma Avaliação de Vieses de Estima por Raça, Gênero e Região em Grandes Modelos de Linguagem em Português Brasileiro. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{melo2026levados,
  title = {{Levados em Consideração: Uma Avaliação de Vieses de Estima por Raça, Gênero e Região em Grandes Modelos de Linguagem em Português Brasileiro}},
  author = {João Lucas Lima {de Melo} and Marlo Souza},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.51/}
}

References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fernanda Assi and Helena Caseli. 2024. Biases in gpt-3.5 turbo model: a case study regarding gender and language. In Anais do XV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 294–305, Porto Alegre, RS, Brasil. SBC. find
  2. Maria Aparecida Baccega. 1998. O estereótipo e as diversidades. Comunicação & Educação, (13):7–14. find
  3. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Associat… find
  4. Camiel J Beukeboom and Christian Burgers. 2019. How stereotypes are shared through language: a review and introduction of the aocial categories and stereotypes communication (scsc) framework. Review of Communication Research, 7:1–37. find
  5. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in nlp. Preprint, arXiv:2005.14050. dataset and multilingual analysis. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computati…
  6. Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of psychology. find
  7. Sumi Cho, Kimberlé Williams Crenshaw, and Leslie McCall. 2013. Toward a field of intersectionality studies: Theory, applications, and praxis. Signs: Journal of women in culture and society, 38(4):785– 810. find
  8. Patricia Hill Collins, Elaini Cristina Gonzaga da Silva, Emek Ergun, Inger Furseth, Kanisha D Bond, and Jone Martínez-Palacios. 2021. Intersectionality as critical social theory: Intersectionality as critical social theory, patricia hill collins, duke university press, 2019. Contemporary Political T… find
  9. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pa… find
  10. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369, Online. Association for Computational Linguisti…
  11. Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating African-American Vernacular English in transformer-based text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)…
  12. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598, Berlin, Germany. Association for Computational Linguistics.
  13. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2021. The state and fate of linguistic diversity and inclusion in the nlp world. Preprint, arXiv:2004.09095.
  14. João Augusto Leite, Diego Silva, Kalina Bontcheva, and Carolina Scarton. 2020. Toxic language detection in social media for Brazilian Portuguese: New Milagros Miceli, Julian Posada, and Tianling Yang. 2022. Studying up machine learning data: Why talk about bias when we mean power? Proc. ACM Hum.-Com… find
  15. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…
  16. Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-pairs: A challenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953–1967, Online. Association for…
  17. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105, Dublin, Ireland. Asso…
  18. Golbery Rodrigues, Danyllo Albuquerque, and Jesualdo Chagas. 2023. Análise de vieses ideológicos em produções textuais do assistente de bate-papo chatgpt. In Anais do IV Workshop sobre as Implicações da Computação na Sociedade, pages 148–155, Porto Alegre, RS, Brasil. SBC. find
  19. Renata Sena, Marlo Souza, Adriana Santana, and João Melo. 2025. Me deixe pensar sobre isso! uma análise do uso de cot para identificar vieses nas respostas de llm para o português brasileiro. In Anais do VI Workshop sobre as Implicações da Computação na Sociedade, pages 105–120, Porto Alegre, RS, Br… find
  20. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024a. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. Preprint, ar-Xiv:2308.03825.
  21. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024b. “Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. In ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM.
  22. Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, and Nanyun Peng. 2021. Revealing persona biases in dialogue systems. Preprint, arXiv:2104.08728. — [arXiv:2104.08728](https://arxiv.org/abs/2104.08728) find
  23. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. Preprint, arXiv:1909.01326.
  24. Fernanda Taso, Valéria Reis, and Fábio Martinez. 2023a. Discriminação algorítmica de gênero: Estudo de caso e análise no contexto brasileiro. In Anais do IV Workshop sobre as Implicações da Computação na Sociedade, pages 13–25, Porto Alegre, RS, Brasil. SBC. find
  25. Fernanda Taso, Valéria Reis, and Fábio Martinez. 2023b. Sexismo no brasil: análise de um word embedding por meio de testes baseados em associação implícita. In Anais do XIV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 53– 62, Porto Alegre, RS, Brasil. SBC. find
  26. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, and 4 others. 2021. E… — Ethical and social risks of harm from Language Models
  27. Langdon Winner. 1980. Do artifacts have politics? Daedalus, 109(1):121–136. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/melo-2026-levados-em-consideracao-uma-avaliacao/