Elamur Research⌕
☰
home / papers / 2026 / Colorism in Multimodal AI: An Empirical Exploration of Socioeconomic Linguistic Bias in Text-to-Image Generation

Colorism in Multimodal AI: An Empirical Exploration of Socioeconomic Linguistic Bias in Text-to-Image Generation

Paper · 2026 · Raj Gaurav Maurya, Vaibhav Shukla and Sreedath Panat

Summary

The paper asks whether income-related wording in prompts pushes text-to-image systems toward lighter or darker skin in the people they depict. It treats colorism as a visible trace of socioeconomic stereotyping. Taking the MST-E dataset as ground truth, the authors first measured how well three commercial VLMs and 60 human raters assign skin tones on the 10-point Monk Skin Tone (MST) scale. GPT-5 mini proved the most dependable labeler and became the automated auditor. Next, they wrote prompts for 210 occupations that varied the person's income framing (affluent or poor) and the degree of prompt constraint. These prompts produced more than 2,500 portraits from three commercial generators. The audit found a consistent pattern: affluent framing yielded lighter faces and poorer framing darker ones. Tighter prompts narrowed this gap only somewhat, and the size of the effect differed between generators. The study links sociological research on colorism with multimodal fairness auditing. It also warns that such cross-modal stereotypes carry risks for hiring, education and digital identity verification.

Key points

  • Annotator benchmark: gemini-2.0-flash-lite-001, gpt-4o-mini and gpt-5-mini were accessed via OpenRouter. Each model received the MST orb chart plus an MST-E image, and each ran three passes over the dataset. A human baseline came from an anonymous web survey with 60 participants.
  • Reference data: MST-E contains 1515 images and 31 videos of 19 subjects. The subjects span the full MST range and were captured under varied lighting, poses and expressions.
  • Bias audit: Portraits came from GPT-5 Image-mini, Gemini-2.5 Flash-Image and Grok-2 Image. The first two showed larger income-driven MST shifts than Grok-2 Image.
  • Stated limitations: The study uses only proprietary models and English-only prompts. Its prompt templates bundle several cues together, and its statistics do not model paired prompts within occupations. Annotators were validated on real photos, which may not transfer to synthetic faces.
  • Resources: Code is released at github.com/RajGM/EACL_VLM_Paper, and the generated image set is publicly shared.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technical University of MunichMunich · Germany62 works · 48,970 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 811 works

Cite

Cite as Raj Gaurav Maurya, Vaibhav Shukla and Sreedath Panat. 2026. Colorism in Multimodal AI: An Empirical Exploration of Socioeconomic Linguistic Bias in Text-to-Image Generation. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.69

@inproceedings{maurya2026colorism,
  title = {{Colorism in Multimodal AI: An Empirical Exploration of Socioeconomic Linguistic Bias in Text-to-Image Generation}},
  author = {Raj Gaurav Maurya, Vaibhav Shukla and Sreedath Panat},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.69},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.69}
}

References

53 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maria Abascal and Denia Garcia. 2022. Pathways to Skin Color Stratification: The Role of Inherited (Dis)Advantage and Skin Color Discrimination in Labor Markets. Sociological Science, 9:346–373. find
  2. Ran Abramitzky, Jacob Conway, Roy Mill, and Luke Stein. 2023. The gendered impacts of perceived skin tone: Evidence from african-american siblings in 1870–1940. Technical report, National Bureau of Economic Research. find
  3. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. find
  4. Alexander Adames. 2023. The Cumulative Effects of Colorism: Race, Wealth, and Skin Tone. Social Forces, 102(2):539–560. find
  5. Jacob Cohen. 1992. A power primer. Psychological Bulletin, 112(1):155–159. find
  6. Nouar AlDahoul, Talal Rahwan, and Yasir Zaki. 2025. Ai-generated faces influence gender stereotypes and racial homogenization. Scientific reports, 15(1):14449. find
  7. Vatsal Baherwani and Joseph James Vincent. 2024. Racial and gender stereotypes encoded into clip representations. In The Second Tiny Papers Track at ICLR 2024. find
  8. Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. 2023. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM conference on… find
  9. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  10. Philip R. Cohen, Marissa A. DiMarco, Rebecca L. Geller, and Leatrice A. Darrisaw. 2023. Colorimetric scale for skin of color: A practical classification scale for the clinical assessment, dermatology management, and forensic evaluation of individuals with skin of color. Cureus, 15(11):e48132. find
  11. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. Preprint, arXiv:2304.05335.
  12. Thomas B. Fitzpatrick. 1988. The validity and practicality of sun-reactive skin types i through vi. Archives of Dermatology, 124(6):869–871. find
  13. Kathleen C Fraser and Svetlana Kiritchenko. 2024. Examining gender and racial bias in large vision– language models using a novel dataset of parallel images. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages… find
  14. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097– 1179.
  15. Mauricio Bucca. 2024. Colorism Revisited: The Effects of Skin Color on Educational and Labor Market Outcomes in the United States. Sociological Science, 11:517–552. find
  16. Leander Girrbach, Stephan Alaniz, Yiran Huang, Trevor Darrell, and Zeynep Akata. 2024. Revealing and reducing gender biases in vision and language assistants (vlas). arXiv:2410.19314. find
  17. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 77–91. PMLR. find
  18. Leander Girrbach, Stephan Alaniz, Genevieve Smith, and Zeynep Akata. 2025. A large scale analysis of gender biases in text-to-image generative models. arXiv:2503.23398. — [arXiv:2503.23398](https://arxiv.org/abs/2503.23398) find
  19. A. Chardon, I. Cretois, and C. Hourseau. 1991. Skin colour typology and suntanning pathways. International Journal of Cosmetic Science, 13(4):191–208. find
  20. Marc Cheong, Ehsan Abedin, Marinus Ferreira, Ritsaart Reimann, Shalom Chalson, Pamela Robinson, Joanne Byrne, Leah Ruppanner, Mark Alfano, and Colin Klein. 2024. Investigating gender and racial biases in dall-e mini images. ACM Journal on Responsible Computing, 1(2):1–20. find
  21. Tavishi Choudhary. 2025. Political bias in large language models: A comparative analysis of chatgpt-4, perplexity, google gemini, and claude. IEEE Access, 13:11341–11379. find
  22. Vishal Gupta and Vinod Kumar Sharma. 2019. Skin typing: Fitzpatrick grading and others. Clinics in Dermatology, 37(5):430–436. The Color of Skin. find
  23. Joni Hersch. 2024. Colorism and immigrant earnings in the United States, 2015–2024. Frontiers in Sociology, 9:1494236. find
  24. John J. Howard, Yevgeniy B. Sirotin, Jerry L. Tipton, and Arun R. Vemury. 2021. Reliability and validity of image-based and self-reported skin phenotype metrics. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(4):550–560. find
  25. ILO. 2008. International Standard Classification of Occupations (ISCO): Concepts and Definitions. ILO Statistics Webpage. Accessed: 2025-01-10. find
  26. Yukun Jiang, Zheng Li, Xinyue Shen, Yugeng Liu, Michael Backes, and Yang Zhang. 2024. Mod-SCAN: Measuring stereotypical bias in large vision-language models from vision and language modalities. arXiv:2410.06967. — [arXiv:2410.06967](https://arxiv.org/abs/2410.06967) find
  27. Muhammad Osto, Iltefat H. Hamzavi, Henry W. Lim, and Indermeet Kohli. 2022. Individual typology angle and fitzpatrick skin phototypes are not equivalent in photodermatology. Photochemistry and Photobiology, 98(1):127–129. find
  28. Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. 2025. Why language models hallucinate. Preprint, arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) find
  29. Matthew A. Painter and Malcolm D. Holmes. 2023. Persistent skin tone and wealth stratification among new immigrants in the United States. Research in Social Stratification and Mobility, 83:100766. find
  30. Aiswarya Konavoor, Raj Abhijit Dandekar, Rajat Dandekar, and Sreedath Panat. 2025. Vision-language models display a strong gender bias. arXiv:2508.11262. — [arXiv:2508.11262](https://arxiv.org/abs/2508.11262) find
  31. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internatio…
  32. L’Oréal. 2024. Inskin: Understanding skin science. L’Oréal Science & Technology Webpage. find
  33. Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2023. Stable bias: Evaluating societal representations in diffusion models. Advances in Neural Information Processing Systems, 36:56338– 56351. find
  34. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. OpenAI. find
  35. Douglas S. Massey and Jennifer A. Martin. 2003. The nis skin color scale. Office of Population Research, Princeton University. find
  36. Joseph L. Rodgers and W. Alan Nicewander. 1988. Thirteen ways to look at the correlation coefficient. Psychological Bulletin, 103(1):59–66. find
  37. Raj Gaurav Maurya, Vaibhav Shukla, Raj Abhijit Dandekar, Rajat Dandekar, and Sreedath Panat. 2025. Simulating misinformation propagation in social networks using large language models. arXiv:2511.10384. — [arXiv:2511.10384](https://arxiv.org/abs/2511.10384) find
  38. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
  39. Vishal Mirza, Rahul Kulkarni, and Aakanksha Jadhav. 2025. Evaluating gender, racial, and age biases in large language models: A comparative analysis of occupational and crime scenarios. In 2025 IEEE Conference on Artificial Intelligence (CAI), pages 244–251. find
  40. Ashutosh Sathe, Prachi Jain, and Sunayana Sitaram. 2024. A unified framework and dataset for assessing societal bias in vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1208–1249, Miami, Florida, USA. Association for Computational Linguistics. find
  41. Ellis P. Monk. 2014. Skin tone stratification among black americans, 2001–2003. Social Forces, 92(4):1313–1337. find
  42. Candice Schumann, Femi Olanubi, Auriel Wright, Ellis Monk, Courtney Heldreth, and Susanna Ricco. 2023. Consensus and subjectivity of skin tone annotation for ml fairness. Advances in Neural Information Processing Systems, 36:30319–30348. find
  43. Ellis P. Monk. 2015. The cost of color: Skin color, discrimination, and health among african-americans. American Journal of Sociology, 121(2):396–444. find
  44. Ellis P. Monk. 2019. The color of punishment: African americans, skin tone, and the criminal justice system. Ethnic and Racial Studies, 42(10):1593–1612. find
  45. Ellis P. Monk. 2019. Monk Skin Tone Scale. Online Resource. find
  46. Yixin Wan, Arjun Subramonian, Anaelia Ovalle, Zongyu Lin, Ashima Suvarna, Christina Chance, Hritik Bansal, Rebecca Pattichis, and Kai-Wei Chang. 2024. Survey of bias in text-to-image generation: Definition, evaluation, and mitigation. arXiv:2404.01030. — [arXiv:2404.01030](https://arxiv.org/abs/2404.01030) find
  47. Kyra Wilson, Sourojit Ghosh, and Aylin Caliskan. 2025. Bias amplification in stable diffusion’s representation of stigma through skin tones and their homogeneity. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(3):2705–2717. find
  48. Ellis P. Monk. 2021a. Colorism and Physical Health: Evidence from a National Survey. Journal of Health and Social Behavior, 62(1):37–52. find
  49. Ellis P. Monk. 2021b. The unceasing significance of colorism: Skin tone stratification in the united states. Daedalus, 150(2):76–90. find
  50. L. Guillermo Woo-Mora. 2026. Unveiling the Cosmic Race: Skin tone and intergenerational economic disparities in Latin America and the Caribbean. Journal of Development Economics, 179:103594. find
  51. Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, and Yi Fang. 2024. Evaluating fairness in large vision-language models across diverse demographic attributes and prompts. arXiv:2406.17974. find
  52. Rhiannon Yetsenga, Rhea Banerjee, Jared Streatfeild, Katherine McGregor, S. Bryn Austin, Belle W.X. Lim, Phillippa C. Diedrichs, Kayla Greaves, Josiemer Mattei, Rebecca M. Puhl, Jaime C. Slaughter-Acey, Iyiola Solanke, Kendrin R. Sonneville, Katrina Velasquez, and Simone Cheung. 2024. The economic a… find
  53. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2025. Siren’s song in the ai ocean: A survey on hallucination in large language models. Computational Linguistics, pages…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maurya-2026-colorism-multimodal-ai-empirical-exploration/