Elamur Research⌕
☰
home / papers / 2025 / Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English

Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English

Paper · 2025 · Runtao Zhou, Guangya Wan, Saadia Gabriel et al.

Summary

The paper asks whether large language models reason as well, and explain themselves as well, when a question is written in African American English (AAE) as when it is written in Standard American English (SAE). The authors build an evaluation pipeline. An LLM rewrites SAE prompts into written AAE, guided by a structured prompt that encodes 11 morphosyntactic features from VALUE and avoids phonetic respellings. Native AAE speakers judged these conversions better than the AAVENUE converter's output. The models then answer the parallel SAE and AAE questions using chain-of-thought and post-hoc explanations. The analysis covers accuracy, consistency across several decoding runs, readability and psychological markers of the explanations. AAE inputs consistently get lower accuracy and simpler reasoning and explanations. The gap is widest in social science and humanities questions. The authors also test prompt-based mitigations, and a combined strategy narrows the gap noticeably. This matters because LLMs are being used to teach and advise, and a quieter drop in explanation quality would hit AAE speakers hardest.

Key points

  • Conversion prompt. The converter prompt lists explicit rules with example sentences for 11 VALUE morphosyntactic features. It deliberately leaves out phonetic changes such as "that" → "dat", which AAVENUE tends to make.
  • Human evaluation of conversions.
    • 100 GPT-4-generated SAE sentences were converted and rated by native AAE speakers recruited via Prolific (12 raters).
    • The new converter won 74–79% of pairwise comparisons against AAVENUE on fluency, coherence, understandability and quality.
    • It also scored 7.97/10 for realism versus 7.62 for AAVENUE.
    • Agreement was measured with Fleiss' κ, and significance was tested with paired binomial tests.
  • Explanation measures. Explanations were scored for readability with FRES (higher means simpler), for psychological expression and for consistency across multiple decoding paths.
  • Mitigation results (mitigation table).
    • With plain prompting, GPT-3.5 on MMLU drops from 66.2 to 59.4 accuracy on AAE. The other model block (GPT-4) drops from 82.5 to 71.8.
    • A combined multi-strategy prompt shrinks these gaps to −2.9 and −4.8 respectively.
    • "Expert Teacher" framing helps both dialects. Cultural-context and dialect-recognition instructions trade a little SAE accuracy for AAE gains.
  • Limitations noted by the authors. Only one dialect is studied, the human evaluation is small, and written text cannot carry prosody.

Interesting ideas

  • I1. Replace a few arbitrary demonstrations with a rule-per-feature prompt, giving each grammatical feature its own linguistically motivated example, so conversions are principled and repeatable.
  • I2. Compare paired inputs that share content but differ only in dialect, which isolates the effect of surface variety on downstream behaviour.
  • I3. Judge the explanation, not only the final answer, using readability and complexity scores. This catches degradations that accuracy alone would hide.
  • I4. Treat agreement across several decoding runs as a stability signal alongside correctness.
  • I5. The habitual "be" in "He be working" marks aspect in its own right, not a sloppy form of SAE, which is a reminder that dialect grammar can carry meaning.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Rule-per-feature prompts could generate AAE or other dialect variants of UD treebank sentences, giving controlled data to test whether the perceptron parser and gparse hold up under dialect shift.
  • I2 (learn): Dialect-swapped minimal pairs make a natural negative control, since any change in the answer exposes form-sensitivity that the meaning pipeline should not have.
  • I3 (answer): The school-test readers could log explanation depth and readability per input variant, flagging cases where the reasoning chain gets thinner although the answer stays right.
  • I4 (learn): This mirrors witness-parser agreement for silver data, and run-to-run disagreement on dialect inputs could mark examples needing teacher review.
  • I5 (frames): Frame building could map invariant habitual "be" to a habitual-aspect feature on the event, rather than discarding it as an agreement error, so the world model records a repeated state.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEnglish · 2,263 worksReasoning · 1,426 works

Cite

Cite as Runtao Zhou, Guangya Wan, Saadia Gabriel, Sheng Li, Alexander J Gates, Maarten Sap and Thomas Hartvigsen. 2025. Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English. arXiv:2503.04099.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2503.04099 · arXiv 2503.04099

@misc{zhou2025disparities,
  title = {{Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English}},
  author = {Runtao Zhou, Guangya Wan, Saadia Gabriel, Sheng Li, Alexander J Gates, Maarten Sap and Thomas Hartvigsen},
  year = {2025},
  doi = {10.48550/arxiv.2503.04099},
  eprint = {2503.04099},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2503.04099}
}

References

68 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Carolyn Temple Adger, Walt Wolfram, and Donna Christian. 2014. Dialects in schools and communities. Routledge. ↗
  2. H. Samy Alim, John R. Rickford, and Arnetha F. Ball. 2016. Raciolinguistics: How Language Shapes Our Ideas About Race. Oxford University Press. find
  3. Michael Argyle and Luo Lu. 1990. The happiness of extraverts. Personality and individual differences, 11(10):1011–1017. ↗
  4. April Baker-Bell. 2020. Linguistic justice: Black language, literacy, identity, and pedagogy. Routledge. find
  5. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. arXiv preprint arXiv:2005.14050. ↗
  6. Su Lin Blodgett and Brendan T. O’Connor. 2017. Racial disparity in natural language processing: A case study of social media african-american english. ArXiv. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R’e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2021. On the opportunities and risks of foundation models. ArXiv. ↗
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901.
  8. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural language inference with natural language explanations. In NeurIPS. ↗
  9. Wendi Cui, Jiaxin Zhang, Zhuohang Li, Lopez Damien, Kamalika Das, Bradley Malin, and Sricharan Kumar. 2024. Dcr-consistency: Divide-conquer-reasoning for consistency evaluation and improvement of large language models. arXiv preprint arXiv:2401.02132. ↗
  10. N. Deas, J. Grieser, S. Kleiner, D. Patton, E. Turcan, and K. McKeown. 2023. Evaluation of african american language bias in natural language generation. In Proceedings of EMNLP. find
  11. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. BOLD: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 862–872. find
  12. Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630. ↗
  13. Rudolf Flesch. 1948. A new readability yardstick. Journal of Applied Psychology, 32(3):221–233. ↗
  14. ME Francis and Roger J Booth. 1993. Linguistic inquiry and word count. Southern Methodist University: Dallas, TX, USA. find
  15. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and Abhishek Kadian et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783. ↗
  16. Lisa J Green. 2002. African American English: a linguistic introduction. Cambridge University Press. ↗
  17. Jessica A Grieser. 2022. The Black side of the river: Race, language, and belonging in Washington, DC. Georgetown University Press. find
  18. Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating african-american vernacular english in transformer-based text generation. Proceedings of EMNLP.
  19. Jeffrey Grogger. 2011. Speech patterns and racial wage inequality. Journal of Human Resources, 46(1):1–25. ↗
  20. Abhay Gupta, Ece Yurtseven, Philip Meng, and Kevin Zhu. 2024. AAVENUE: Detecting LLM biases on NLU tasks in AAVE via a novel benchmark. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 327–333, Miami, Florida, USA. Association for Computational Linguistics.
  21. Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2023. Bias runs deep: Implicit reasoning biases in persona-assigned llms. arXiv preprint arXiv:2311.04892. ↗
  22. Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988, 1. ↗
  23. Martin N Hebart and Guido Hesselmann. 2012. What visual information is processed in the human dorsal stream? Journal of Neuroscience, 32(24):8107– 8109. ↗
  24. Paula Helm, Gábor Bella, Gertraud Koch, and Fausto Giunchiglia. 2024. Diversity and language technology: how language modeling bias causes epistemic injustice. Ethics and Information Technology, 26(1):8. ↗
  25. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. ↗
  26. V. Hofmann, P. R. Kalluri, D. Jurafsky, and S. King. 2024. Dialect prejudice predicts ai decisions about people’s character, employability, and criminality. arXiv preprint. ↗
  27. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, and Devendra Singh Chaplot et al. 2023a. Mistral 7b. Preprint, arXiv:2310.06825. ↗
  28. Albert Q Jiang et al. 2023b. Generating with confidence: Uncertainty quantification for black-box large language models. find
  29. Taylor Jones, Jessica Rose Kalbfeld, Ryan Hancock, and Robin Clark. 2019. Testifying while black: An experimental study of court reporter accuracy in transcription of african american english. Language, 95(2):e216–e252. ↗
  30. Anjali Kantharuban, Jeremiah Milbauer, Emma Strubell, and Graham Neubig. 2024. Stereotype or personalization? user identity biases chatbot recommendations. arXiv preprint arXiv:2410.05613. ↗
  31. Selim Kılıç. 2015. Kappa testi. Journal of mood disorders, 5(3):142–144. find
  32. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213. ↗
  33. Eliza Kosoy et al. 2023. Understanding the role of large language models in education. find
  34. Emanuele La Malfa et al. 2024. Language-models-asa-service: Overview of a new paradigm and its challenges. Journal of Artificial Intelligence Research, 80:1497–1523. find
  35. Yingji Li, Mengnan Du, Rui Song, Xin Wang, and Ying Wang. 2024. A survey on fairness in large language models. Preprint, arXiv:2308.10149. ↗
  36. Yuxuan Li, Hirokazu Shirado, and Sauvik Das. 2025. Actions speak louder than words: Agent decisions reveal implicit biases in language models. Preprint, arXiv:2501.17420. ↗
  37. Fangru Lin, Shaoguang Mao, Emanuele La Malfa, Valentin Hofmann, Adrian de Wynter, Jing Yao, Si-Qing Chen, Michael Wooldridge, and Furu Wei. 2024. One language, many gaps: Evaluating dialect fairness and robustness of large language models in reasoning tasks. arXiv preprint arXiv:2410.11005. ↗
  38. Haoyan Luo and Lucia Specia. 2024. From understanding to utilization: A survey on explainability for large language models. Preprint, arXiv:2401.12874. ↗
  39. Kyle Mahowald et al. 2024. Dissociating language model knowledge and capabilities through instances of minimal pairs. find
  40. Douglas S Massey and Garvey Lundy. 2001. Use of black english and racial discrimination in urban housing markets. Urban Affairs Review, 36(4):452–469. ↗
  41. Dan Milmo. 2023. Chatgpt passes 100 million users, making it the fastest-growing app in history. The Guardian. find
  42. Eric Mitchell et al. 2023. Debate as a diagnostic tool for understanding large language model capabilities. find
  43. P. Mondorf and B. Plank. 2024. Beyond accuracy: Evaluating the reasoning behavior of large language models - a survey. COLM 2024 Conference Proceedings. ↗
  44. Stanislaw Jerzy Niepostyn and Wiktor Bohdan Daszczuk. 2023. Entropy as a measure of consistency in software architecture. Entropy, 25(2):328. ↗
  45. OpenAI. 2024. GPT-4 technical report. arXiv preprint arXiv:2303.08774. ↗
  46. Luigi Pastore and Sara Dellantonio. 2016. Modelling scientific un/certainty. why argumentation strategies trump linguistic markers use. In Model-Based Reasoning in Science and Technology: Logical, Epistemological, and Cognitive Issues, pages 137–164. Springer. ↗
  47. John R Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing rachel jeantel (and other vernacular speakers) in the courtroom and beyond. Language, 92(4):948–988. ↗
  48. Richard D Rieke and Randall K Stutman. 2022. Communication in legal advocacy. Univ of South Carolina Press. ↗
  49. M. Sap, D. Card, S. Gabriel, Y. Choi, and N. A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of ACL.
  50. Guijin Son, Sangwon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. 2024. Multi-task inference: Can large language models follow multiple instructions at once? Preprint, arXiv:2402.11597. ↗
  51. Arthur Spears. 1998. African-american language use: Ideology and so-called obscenity. pages 226–250. find
  52. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. ↗
  53. Ian Stewart. 2014. Now we stronger than ever: African-American English syntax in Twitter. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 31–37, Gothenburg, Sweden. Association for Computational Linguistics. ↗
  54. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54. ↗
  55. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, and Surya Bhupatiraju et al. 2024. Gemma: Open models based on gemini research and technology. Preprint, arXiv:2403.08295. ↗
  56. Thiemo Wambsganss, Xiaotian Su, Vinitra Swamy, Seyed Neshaei, Roman Rietsche, and Tanja Käser. 2023. Unraveling downstream gender bias from large language models: A study on AI educational writing assistance. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10275– 10288, Singapore. Association for Computational Linguistics. find
  57. Guangya Wan, Yunsheng Lu, Yuqi Wu, Mengxuan Hu, and Sheng Li. 2025. Large language models for causal discovery: Current landscape and future directions. Preprint, arXiv:2402.11068. ↗
  58. Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. 2024. Dynamic self-consistency: Leveraging reasoning paths for efficient llm sampling. Preprint, arXiv:2408.17017. ↗
  59. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In NeurIPS.
  60. Yuqi Wu, Guangya Wan, Jingjing Li, Shengming Zhao, Lingfeng Ma, Tianyi Ye, Ion Pop, Yanbo Zhang, and Jie Chen. 2025. Proai: Proactive multi-agent conversational ai with structured knowledge base for psychiatric diagnosis. Preprint, arXiv:2502.20689. ↗
  61. Zhen Wu, Ritam Dutt, and Carolyn Rose. 2024. Evaluating large language models on social signal sensitivity: An appraisal theory approach. In Proceedings of the 1st Human-Centered Large Language Modeling Workshop, pages 67–80, TBD. ACL. find
  62. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, and Bowen Yu et al. 2024. Qwen2 technical report. Preprint, arXiv:2407.10671. ↗
  63. Taha Yasseri, András Kornai, and János Kertész. 2012. A practical approach to language complexity: a wikipedia case study. PloS one, 7(11):e48386. ↗
  64. Yuxuan Ye, Edwin Simpson, and Raul Santos Rodriguez. 2024. Using similarity to evaluate factual consistency in summaries. arXiv preprint arXiv:2409.15090. ↗
  65. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2024. A survey of large language models. Preprint, arXiv:2303.18223. ↗
  66. Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, and David Jurgens. 2024. When ”a helpful assistant” is not really helpful: Personas in system prompts do not improve performances of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15126–15154, Miami, Florida, USA. Association for Computational Linguistics. find
  67. Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, and Maarten Sap. 2023. Cobra frames: Contextual reasoning about effects and harms of offensive statements. In Findings of ACL. find
  68. Caleb Ziems, Jiaao Chen, Camille Harris, Jessica Anderson, and Diyi Yang. 2022. VALUE: Understanding dialect disparity in NLU. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 3701–3720.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2025-disparities-llm-reasoning-accuracy-explanations/