Elamur Research⌕
☰
home / papers / 2024 / Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Paper · 2024 · Eve Fleisig, Genevieve Smith, Madeline Bossi et al.

Summary

This study asks whether ChatGPT treats speakers of different English dialects unequally. The authors gathered informal, native-speaker-written text in ten varieties: two prestige "standard" ones (SAE, SBE) and eight minoritized ones (AAE, Indian, Irish, Jamaican, Kenyan, Nigerian, Scottish, Singaporean English). They fed these messages to GPT-3.5 Turbo and GPT-4 and ran two studies.

In the first, annotators marked distinctive dialect features in inputs and outputs to see how many survive into the model's reply. In the second, native speakers rated replies on axes such as stereotyping, demeaning tone, condescension, warmth and comprehension. Replies were collected both by default and when the model was told to imitate the input dialect.

The models drift toward standard English. Minoritized varieties get replies judged more stereotyping, more demeaning, more condescending and less comprehending. Imitation makes things worse, and GPT-4, while warmer and better at understanding, stereotypes even more. The work shows that widely used assistants can reinforce existing linguistic discrimination.

Key points

  • Data sources:
    • Nigerian, Jamaican, Indian, Irish and Kenyan English come from the social-letters part of the International Corpus of English (ICE).
    • SAE and SBE come from Reddit city subreddits (Zhang, 2023).
    • AAE comes from Blodgett et al. (2018).
    • Scottish English comes from the SCOTS corpus correspondence subset.
    • Singaporean English comes from CoSEM text messages.
    • About 50 messages were used per variety.
  • Feature set: for each variety, ten prominent morphosyntactic or lexical features were chosen from linguistic literature. Output orthography was also labeled as American, British, or either.
  • Feature retention in GPT-3.5 replies:
    • SAE 78% and SBE 72%.
    • Indian 16%, Nigerian 13%, Kenyan 10%.
    • Irish 4%, AAE, Scottish and Singaporean 3% each, Jamaican 2%.
    • Among minoritized varieties, retention tracks speaker population size.
  • Native-speaker ratings: compared with standard varieties, replies to minoritized ones were rated:
    • 19% worse on stereotyping;
    • 25% worse on demeaning content;
    • 9% worse on comprehension;
    • 15% worse on condescension.
  • Three conditions were rated by native speakers recruited via Prolific: GPT-3.5 default, GPT-3.5 imitating, and GPT-4 imitating. GPT-4 raised stereotyping by about 18% while improving warmth, friendliness and comprehension.

Interesting ideas

  • I1. Measure dialect preservation by counting a fixed set of diagnostic features in the input and the output, giving a simple per-variety retention rate that is easy to inspect.
  • I2. Build a small per-variety inventory of hallmark constructions taken from descriptive grammars, such as Indian English yes-no questions without subject-verb inversion.
  • I3. Track spelling convention (American, British, or undecidable) as a separate output label, distinct from grammar features.
  • I4. Rate outputs on several separate harm and quality axes across conditions, since a stronger model can improve some axes while getting worse on another.
  • I5. Use native speakers both to score comprehension and to give free-text reactions, which exposes failure modes like undue formality or exaggerated imitation that metrics miss.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Apply the same input-vs-output feature count to the text → graph → text compressor to check which dialect markers survive the round trip.
  • I2 (tree): Such a hand-picked construction list could serve as a targeted test suite for the parsers and for UD dialect conversion, revealing where trees break on non-standard syntax.
  • I3 (tokens): The lemmatizer and morphology in crate en could tag orthographic variety (analyse/analyze, favour/favor) explicitly, so normalization stays explainable rather than silently Americanizing.
  • I4 (learn): Evaluation should report per-axis scores per dialect slice instead of one aggregate, so a parser or teacher upgrade that hurts one group is not hidden by overall gains.
  • I5 (answer): Readers answering fable or school-test questions could be checked on dialect-written inputs, with native-speaker judgments of whether the answer actually understood the text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, BerkeleyBerkeley · United States175 works · 227,440 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Eve Fleisig, Genevieve Smith, Madeline Bossi, Ishita Rustagi, Xavier Yin and Dan Klein. 2024. Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.emnlp-main.750

@inproceedings{fleisig2024linguistic,
  title = {{Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination}},
  author = {Eve Fleisig, Genevieve Smith, Madeline Bossi, Ishita Rustagi, Xavier Yin and Dan Klein},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.750},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.750}
}

References

44 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Carolyn Temple Adger, Walt Wolfram, and Donna Christian. 2014. Dialects in Schools and Communities. Routledge. ↗
  2. M.A. Alo and Rajend Mesthrie. 2004. Nigerian English: Morphology and syntax. In A Handbook of Varieties of English, pages 323–339. De Gruyter Mouton. find
  3. Jean Anderson, Dave Beavan, and Christian Kay. 2007. Scots: Scottish corpus of texts and speech. In Creating and Digitizing Language Corpora: Volume 1: Synchronic Databases, pages 17–34. Springer. ↗
  4. April Baker-Bell. 2020. Linguistic justice. NCTE-Routledge Research Series. Routledge, London, England. find
  5. John Baugh. 2005. Linguistic profiling. In Black linguistics, pages 167–180. Routledge. find
  6. Kenneth Beare. 2019. American English to British English vocabulary. find
  7. Emily M. Bender. 2019. The #BenderRule: On naming the languages we study and why it matters. find
  8. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency parsing for African-American and mainstream American English. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics. ↗
  9. David Britain. 2007. Grammatical variation in England. In David Britain, editor, Language in the British Isles, pages 75–104. Cambridge University Press. ↗
  10. Alfred Buregeya. 2013. Kenyan English. In The Mouton World Atlas of Variation in English, pages 466– 474. De Gruyter Mouton. find
  11. Karen P. Corrigan. 2013. The Atlantic Archipelago of the British Isles. In The Oxford Handbook of World Englishes, pages 335–370. Oxford University Press. ↗
  12. Nicholas Deas, Jessica Grieser, Shana Kleiner, Desmond Patton, Elsbeth Turcan, and Kathleen McKeown. 2023. Evaluation of African American language bias in natural language generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6805– 6824, Singapore. Association for Computational Linguistics. find
  13. Dictionaries of the Scots Language. 2022. Dictionaries of the Scots Language. find
  14. Jeffrey L. Kallen. 2013. Irish English Volume 2: The Republic of Ireland. De Gruyter Mouton. ↗
  15. Paul Kerswill. 2007. Standard and non-standard English. In David Britain, editor, Language in the British Isles, pages 34–51. Cambridge University Press. ↗
  16. Sharese King. 2020. From african american vernacular english to african american language: Rethinking the study of race and language in african americans’ speech. Annual Review of Linguistics, 6(1):285–300. ↗
  17. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689. ↗
  18. Anna Drożdżowicz and Yael Peled. 2024. The complexities of linguistic discrimination. Philosophical Psychology, page 1–24. ↗
  19. William Labov. 2006. The social stratification of English in New York city. Cambridge University Press. ↗
  20. Penelope Eckert and John R Rickford, editors. 2009. Style and Sociolinguistic Variation. Cambridge University Press, Cambridge, England. find
  21. Jakob R. E. Leimgruber. 2013. Singapore English: Structure, variation, and usage. Cambridge University Press. find
  22. Ravinder Gargesh and Pingali Sailaja. 2013. South Asia. In The Oxford Handbook of World Englishes, pages 425–447. Oxford University Press. find
  23. Lisa Lim. 2013. Southeast Asia. In The Oxford Handbook of World Englishes, pages 448–471. Oxford University Press. find
  24. Wilkinson D W Gonzales, Jakob Leimgruber, Mie Hiramoto, and Junjie Lim. 2024. The corpus of singapore english messages (cosem). find
  25. Rosina L. Lippi-Green. 1994. Accent, standard language ideology, and discriminatory pretext in the courts. Language in Society, 23:166. ↗
  26. Lisa J. Green. 2002. African American English: A linguistic introduction. Cambridge University Press. ↗
  27. Joshua L Martin and Kevin Tang. 2020. Understanding racial disparities in automatic speech recognition: The case of habitual" be". In Interspeech, pages 626–630. find
  28. Sidney Greenbaum and Gerald Nelson. 1996. The international corpus of english (ice) project. World Englishes, 15(1):3–15. ↗
  29. Ulrike Gut. 2013. English in West Africa. In The Oxford Handbook of World Englishes, pages 491– 507. Oxford University Press. find
  30. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. Dialect prejudice predicts ai decisions about people’s character, employability, and criminality. Preprint, arXiv:2403.00742. ↗
  31. M. Huber and K. Dako. 2008. Ghanaian English: Morphology and syntax. In Rajend Mesthrie, editor, Varieties of English, volume 3. Mouton de Gruyter, Berlin. ↗
  32. Marianne Hundt and Ulrike Gut. 2012. Varieties of English Around the World. John Benjamins Publishing Company, Amsterdam. find
  33. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics.
  34. Joshua L Martin and Kelly Elizabeth Wright. 2023. Bias in automatic speech recognition: The case of african american language. Applied Linguistics, 44(4):613– 630. ↗
  35. Robert McColl Millar. 2007. Northern and Insular Scots. Edinburgh University Press. ↗
  36. Mohamad Moslimani, Christine Tamir, Abby Budiman, Luis Noe-Bustamante, and Lauren Mora. 2024. Facts about the u.s. black population. find
  37. National Records of Scotland. 2024. Mid-year population estimates for scotland in 2022. find
  38. Julia Nee, Genevieve Macfarlane Smith, Alicia Sheares, and Ishita Rustagi. 2022. Linguistic justice as a framework for designing, developing, and managing natural language processing tools. Big Data & Society, 9(1):205395172210909. ↗
  39. Northern Ireland Statistics and Research Agency. 2021. 2021 census. find
  40. Oxford English Dictionary. 2023. Introduction to Indian English. find
  41. John R Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing rachel jeantel (and other vernacular speakers) in the courtroom and beyond. Language, pages 948–988. American Variety Input Change SAE 47% +5% SBE 4% +29% AAE 2% +9% Indian 0% +36% Irish 1% +26% Jamaican 1% +22% Kenyan 4% +34% Nigerian 2% +43% Scottish 0% +26% Singaporean 1% +13% Michael J Ryan, William Held, and Diyi Yang. 2024. Unintended impacts of llm alignment on global representation. arXiv preprint arXiv:2402.15018. ↗
  42. Andrea Sand. 2013. Jamaican English. In The Mouton World Atlas of Variation in English, pages 210–221. De Gruyter Mouton. find
  43. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668–1678.
  44. Josef Schmied. 2013. East African English. In The Oxford Handbook of World Englishes, pages 472– 490. Oxford University Press. Orthography British Input Change 0% -2% 45% -40% 0% 0% 59% -50% 58% -56% 13% -13% 34% -30% 22% -21% 67% -63% 28% -22% Either Input Change 50% -3% 51% +11% 98% -9% 41% +14% 40% +30% 86% -9% 62% -4% 76% -22% 33% +37% 71% +9% *Table 2: Orthographic changes in inputs and GPT-3.5 outputs.* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fleisig-2024-linguistic-bias-chatgpt-language-models/