Elamur Research⌕
☰
home / papers / 2024 / Bias in Large Language Models: Origin, Evaluation, and Mitigation

Bias in Large Language Models: Origin, Evaluation, and Mitigation

Paper · 2024 · Yufei Guo, Muzhe Guo, Juntao Su et al.

Summary

LLMs now drive much of modern NLP, but they absorb and can amplify social biases. This is risky in high-stakes deployments. Guo et al. address the fragmented state of research on this problem by consolidating it into a single review framework rather than proposing a new model or dataset. They first trace where bias originates. They distinguish intrinsic bias, which stems from training data, collection practices, language contexts, tokenization and word embeddings, from extrinsic bias, which appears in downstream NLU and NLG tasks. Next, they survey detection techniques grouped by the level they examine (data, model internals, generated outputs) and add approaches that depend on human judgment. Debiasing methods are organized by the stage at which they act (pre-model, intra-model, post-model), followed by a critical comparison of their effectiveness and drawbacks. The review closes with ethical and legal issues, using healthcare and criminal justice as examples. Its main contribution is a structured reference for anyone studying or reducing bias in LLMs.

Key points

  • Bias origins are split into intrinsic (four sources: training data, data collection methods, language contexts, tokenization/word embeddings) and extrinsic (NLU vs. NLG tasks).
  • Evaluation methods fall into four families: data-level, model-level, output-level, and human-involved.
  • Mitigation follows a three-stage taxonomy (pre-model, intra-model, post-model debiasing) and includes a dedicated comparative section (5.4).
  • Harms of biased LLMs are discussed with real-world examples from healthcare and criminal justice, along with legal challenges.
  • The paper is available as arXiv:2411.10915 (v1 2024, v2 May 2026) and was published in Electronics 15(9), 1824 (2026). Most authors are from George Washington University's Department of Statistics. The excerpt provided (abstract and contents only) names no specific benchmarks or numeric results.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓George Washington UniversityWashington · United States51 works · 8,148 citations🎓Pennsylvania State UniversityState College · United States56 works · 5,360 citations🎓University of ConnecticutStorrs · United States11 works · 357 citations

Also on the byline: College of Saint Elizabeth.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yufei Guo, Muzhe Guo, Juntao Su, Zhou Yang, Mengqiu Zhu, Hongfei Li, Mengyang Qiu and Shuo Shuo Liu. 2024. Bias in Large Language Models: Origin, Evaluation, and Mitigation. arXiv:2411.10915.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3390/electronics15091824 · arXiv 2411.10915

@misc{guo2024bias,
  title = {{Bias in Large Language Models: Origin, Evaluation, and Mitigation}},
  author = {Yufei Guo, Muzhe Guo, Juntao Su, Zhou Yang, Mengqiu Zhu, Hongfei Li, Mengyang Qiu and Shuo Shuo Liu},
  year = {2024},
  doi = {10.3390/electronics15091824},
  eprint = {2411.10915},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3390/electronics15091824}
}

References

132 references, 52 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Keskar, N.S.; McCann, B.; Varshney, L.R.; Xiong, C.; Socher, R. CTRL: A Conditional Transformer Language Model for Controllable Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China, 3–7 November 2019; pp. 111–129.
  2. Yang, Y.; Yuan, Y.; Liu, L. FinBERT: A Pretrained Language Model for Financial Communications. arXiv 2020, arXiv:2006.08097. find
  3. Okonkwo, C.W.; Ade-Ibijola, A. The role of artificial intelligence in education: Prospects and challenges. J. Educ. Technol. Soc. 2023, 26, 1–12. find
  4. Jiang, L.; Yang, M.; Li, X. Smart Music Player: User-Adaptive Music Recommendation System. find
  5. IEEE Trans. Multimed. 2020, 22, 666–675. find
  6. Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 2020, 33, 1877–1901.
  7. Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018, arXiv:1810.04805.
  8. Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; Zettlemoyer, L. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin…
  9. Nallapati, R.; Zhou, B.; Gulcehre, C.; Xiang, B. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv 2016, arXiv:1602.06023.
  10. Zhang, L.; Wang, S.; Liu, B. Deep Learning for Sentiment Analysis: A Survey. IEEE Trans. find
  11. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P.J. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach.
  12. Bender, E.M.; Gebru, T.; McMillan-Major, A.; Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual, 3–10 March 2021; pp. 610–623. find
  13. Blodgett, S.L.; Barocas, S.; Daumé, H., III; Wallach, H. Language (Technology) is Power: A Critical Survey of “Bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; Jurafsky, D., Chai, J., Schluter, N., Tetreault, J., Eds.; Association for Computat… — Language (Technology) is Power: A Critical Survey of “Bias” in NLP
  14. Rajkomar, A.; Hardt, M.; Howell, M.D.; Corrado, G.; Chin, M.H. Ensuring Fairness in Machine Learning to Advance Health Equity. Ann. Intern. Med. 2018, 169, 866–872. find
  15. Angwin, J.; Larson, J.; Mattu, S.; Kirchner, L. Machine Bias. ProPublica 2016, 23, 139–159. find
  16. Chen, M.; Ma, Z.; Hannak, A.; Wilson, C. My Fair LADY: Detecting and Mitigating Bias in Job Advertisements. In Proceedings of the 2018 World Wide Web Conference, Lyon, France, 23–27 April 2018; pp. 991–1000. find
  17. Caliskan, A.; Bryson, J.J.; Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science 2017, 356, 183–186. find
  18. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed.; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2009. find
  19. Akaike, H. A New Look at the Statistical Model Identification. IEEE Trans. Autom. Control 1974, 19, 716–723. https://doi.org/10.1109/TAC.1974.1100705. — [doi:10.1109/tac.1974.1100705](https://doi.org/10.1109/tac.1974.1100705) ↗
  20. Heckman, J.J. Sample Selection Bias as a Specification Error. Econometrica 1979, 47, 153–161. https://doi.org/10.2307/1912352. — [doi:10.2307/1912352](https://doi.org/10.2307/1912352) ↗
  21. Vardi, Y. Nonparametric Estimation in the Presence of Length Bias. Ann. Stat. 1982, 10, 616–620. https://doi.org/10.1214/aos/1176345802. — [doi:10.1214/aos/1176345802](https://doi.org/10.1214/aos/1176345802) ↗
  22. Tibshirani, R. Regression Shrinkage and Selection via the Lasso. J. R. Stat. Soc. Ser. B (Methodol.) 1996, 58, 267–288. find
  23. Nemes, S.; Jonasson, J.M.; Genell, A.; Steineck, G. Bias in odds ratios by logistic regression modelling and sample size. BMC Med. Res. Methodol. 2009, 9, 56. https://doi.org/10.1186/1471-2288-9-56. — [doi:10.1186/1471-2288-9-56](https://doi.org/10.1186/1471-2288-9-56) ↗
  24. Li, H.; Qian H.; Li, C.T.; Hou, K. Issues in cox proportional hazards model with unequal randomization. J. Biopharm. Stat. 2026, 36, 330–335. https://doi.org/10.1080/10543406.2024.2418139. — [doi:10.1080/10543406.2024.2418139](https://doi.org/10.1080/10543406.2024.2418139) ↗
  25. Rubin, D.B. Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. J. Educ. Psychol. 1974, 66, 688–701. https://doi.org/10.1037/h0037350. — [doi:10.1037/h0037350](https://doi.org/10.1037/h0037350) ↗
  26. Schick, T.; Udupa, S.; Schütze, H. Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP. Trans. Assoc. Comput. Linguist. 2021, 9, 1408–1424. https://doi.org/10.1162/tacl_a_00434. ↗
  27. Sun, T.; Gaut, A.; Tang, S.; Huang, Y.; ElSherief, M.; Zhao, J.; Mirza, D.; Belding, E.; Chang, K.W.; Wang, W.Y. Mitigating gender bias in natural language processing: Literature review. arXiv 2019, arXiv:1906.08976.
  28. Bolukbasi, T.; Chang, K.W.; Zou, J.Y.; Saligrama, V.; Kalai, A.T. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv. Neural Inf.
  29. Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; Chang, K.W. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, LA, USA, 1–6 June 2018; pp. 15–20.
  30. Hovy, D.; Prabhumoye, S. Five Sources of Bias in Natural Language Processing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Online, 1–6 find
  31. Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; Smith, N.A. The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July–2 August 2019; pp. 1668–1678.
  32. Kiritchenko, S.; Mohammad, S.M. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. arXiv 2018, arXiv:1805.04508.
  33. Obermeyer, Z.; Powers, B.; Vogeli, C.; Mullainathan, S. Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations. Science 2019, 366, 447–453. find
  34. Pariser, E. The Filter Bubble: How the New Personalized Web is Changing What We Read and How We Think; Penguin: London, UK, 2011. find
  35. Sap, M.; Gabriel, S.; Qin, L.; Jurafsky, D.; Smith, N.A.; Choi, Y. Social Bias Frames: Reasoning about Social and Power Implications of Language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July–2 August 2019; pp. 5477–5483.
  36. Sheng, E.; Chang, K.W.; Natarajan, P.; Peng, N. Societal Biases in Language Generation: Progress and Challenges. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Online, 1–6 August 2021; pp. 4275–4293.
  37. Nadeem, M.; Bethke, A.; Reddy, S. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Onli…
  38. Hewitt, J.; Manning, C.D. A Structural Probe for Finding Syntax in Word Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, MN, USA, 2–7 June 2019; pp. 4129–4138.
  39. Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; Chang, K.W. Learning Gender-Neutral Word Embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018; pp. 4847–4853.
  40. Liang, P.P.; Zheng, L.; Salakhutdinov, R.; Morency, L.P. Towards Debiasing Sentence Representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 5–10 July 2020; pp. 5502–5515.
  41. Mehrabi, N.; Morstatter, F.; Saxena, N.; Lerman, K.; Galstyan, A. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv. 2021, 54, 1–35.
  42. Navigli, R.; Conia, S.; Ross, B. Biases in Large Language Models: Origins, Inventory, and Discussion. ACM J. Data Inf. Qual. 2023, 15, 1–21. find
  43. Gallegos, I.O.; Rossi, R.A.; Barrow, J.; Tanjim, M.M.; Kim, S.; Dernoncourt, F.; Yu, T.; Zhang, R.; Ahmed, N.K. Bias and fairness in large language models: A survey. Comput. Linguist. 2024, 50, 1097–1179.
  44. Doan, T.V.; Wang, Z.; Nguyen, M.N.; Zhang, W. Fairness in Large Language Models in three hours. In Proceedings of the 33rd ACM International Conference on Information & Knowledge Management, Boise, ID, USA, 21–25 October 2024. find
  45. May, C.; Wang, A.; Bordia, S.; Bowman, S.R.; Rudinger, R. On measuring social biases in sentence encoders. arXiv 2019, arXiv:1903.10561.
  46. Pagano, T.P.; Loureiro, R.B.; Lisboa, F.V.; Peixoto, R.M.; Guimarães, G.A.; Cruz, G.O.; Araujo, M.M.; Santos, L.L.; Cruz, M.A.; Oliveira, E.L.; et al. Bias and unfairness in machine learning models: A systematic review on datasets, tools, fairness metrics, and identification and mitigation methods.… find
  47. Ray, P.P. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things -Cyber-Phys. Syst. 2023, 3, 121–154. find
  48. Goldfarb-Tarrant, S. Fairness in Transfer Learning for Natural Language Processing. Ph.D. Thesis, Institute for Language, Cognition and Computation, School of Informatics, University of Edinburgh, Edinburgh, UK, 2024. Available online: https://hdl.handle.net/1842/41857 ↗
  49. (accessed on 15 November 2024). find
  50. Goldman, J.; Tsotsos, J.K. Statistical Challenges with Dataset Construction: Why You Will Never Have Enough Images. arXiv 2024, arXiv:2408.11160. — [arXiv:2408.11160](https://arxiv.org/abs/2408.11160) find
  51. Das, D.; De Langis, K.; Martin, A.; Kim, J.; Lee, M.; Kim, Z.M.; Hayati, S.; Owan, R.; Hu, B.; Parkar, R.; et al. Under the surface: Tracking the artifactuality of llm-generated data. arXiv 2024, arXiv:2401.14698. — [arXiv:2401.14698](https://arxiv.org/abs/2401.14698) find
  52. Alvero, A.; Lee, J.; Regla-Vargas, A.; Kizilec, R.; Joachims, T.; Antonio, A.L. Large Language Models, Social Demography, and Hegemony: Comparing Authorship in Human and Synthetic Text. J. Big Data 2024, 11, 138. find
  53. UNESCO; IRCAI. Challenging Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models. 2024; 20p. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000388971 (accessed on 15 November 2024). ↗
  54. Ahmad, A.; Bhattacharyya, P. Bias in Language Models: A Survey. CFILT, Indian Institute of Technology Bombay, Mumbai, India, 2024. Available online: https://www.cfilt.iitb.ac.in/resources/surveys/2024/Bias_Survey.pdf (accessed on 15 September 2024). ↗
  55. Talat, Z.; Névéol, A.; Biderman, S.; Clinciu, M.; Dey, M.; Longpre, S.; Luccioni, S.; Masoud, M.; Mitchell, M.; Radev, D.; et al. You reap what you sow: On the challenges of bias evaluation under multilingual settings. In Proceedings of the BigScience Episode\# 5–Workshop on Challenges & Perspectives…
  56. Mukherjee, A.; Caliskan, A.; Zhu, Z.; Anastasopoulos, A. Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning. In Proceedings of the 2024 find
  57. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Mexico City, Mexico, 16–21 June 2024; pp. 6398–6415. find
  58. Lee, B.; Aiyappa, R.; Ahn, Y.Y.; Kwak, H.; An, J. Neural embedding of beliefs reveals the role of relative dissonance in human decision-making. arXiv 2024, arXiv:2408.07237. — [arXiv:2408.07237](https://arxiv.org/abs/2408.07237) find
  59. Shi, W.; Li, R.; Zhang, Y.; Ziems, C.; Horesh, R.; de Paula, R.A.; Yang, D. Culturebank: An online community-driven knowledge base towards culturally aware language technologies. arXiv 2024, arXiv:2404.15238.
  60. Naous, T.; Ryan, M.J.; Ritter, A.; Xu, W. Having beer after prayer? Measuring cultural bias in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, 11–16 August 2024; pp. 16366–16393.
  61. Wang, W.; Jiao, W.; Huang, J.; Dai, R.; Huang, J.T.; Tu, Z.; Lyu, M. Not all countries celebrate thanksgiving: On the cultural dominance in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, 11…
  62. Salinas, A.; Penafiel, L.; McCormack, R.; Morstatter, F. “Im not Racist but...”: Discovering Bias in the Internal Knowledge of Large Language Models. arXiv 2023, arXiv:2310.08780.
  63. Bussaja, J. Evaluating Racial Bias in Large Language Models: The Necessity for “SMOKY”. Int. J. Soft Comput. (IJSC) 2024, 15 , 1–15. https://doi.org/10.5121/ijsc.2024.15301 — [doi:10.5121/ijsc.2024.15301](https://doi.org/10.5121/ijsc.2024.15301) ↗
  64. Kuntz, J.B.; Silva, E.C. Who Authors the Internet? Analyzing Gender Diversity in ChatGPT-3 Training Data; University of Pittsburgh: Pittsburgh, PA, USA, 2023. find
  65. Qu, Y.; Wang, J. Performance and biases of Large Language Models in public opinion simulation. find
  66. Humanit. Soc. Sci. Commun. 2024, 11, 1–13. find
  67. Kotek, H.; Dockum, R.; Sun, D. Gender bias and stereotypes in large language models. In Proceedings of the ACM Collective Intelligence Conference, Delft, The Netherlands, 6–9 find
  68. Dwivedi, S.; Ghosh, S.; Dwivedi, S. Breaking the bias: Gender fairness in LLMs using prompt engineering and in-context learning. Rupkatha J. Interdiscip. Stud. Humanit. 2023, 15, 1–18. de Sá, J.M.C.; Da Silveira, M.; Pruski, C. Semantic Change Characterization with LLMs using Rhetorics. arXiv 2024,…
  69. Petrov, A.; La Malfa, E.; Torr, P.; Bibi, A. Language model tokenizers introduce unfairness between languages. Adv. Neural Inf. Process. Syst. 2023, 36, 36963–36990.
  70. Ahia, O.; Kumar, S.; Gonen, H.; Kasai, J.; Mortensen, D.R.; Smith, N.A.; Tsvetkov, Y. Do all languages cost the same? tokenization in the era of commercial language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6–10 December 2023; pp. 9…
  71. Ovalle, A.; Mehrabi, N.; Goyal, P.; Dhamala, J.; Chang, K.W.; Zemel, R.; Galstyan, A.; Pinter, Y.; Gupta, R. Tokenization matters: Navigating data-scarce tokenization for gender inclusive language technologies. In Proceedings of the Findings of the Association for Computational Linguistics: NAACL 20… find
  72. Garg, N.; Schiebinger, L.; Jurafsky, D.; Zou, J. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc. Natl. Acad. Sci. USA 2018, 115, E3635–E3644. find
  73. Rakshit, A.; Singh, S.; Keshari, S.; Chowdhury, A.G.; Jain, V.; Chadha, A. From prejudice to parity: A new approach to debiasing large language model word embeddings. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, United Arab Emirates, 19–24 January 2025… find
  74. Li, Y.; Du, M.; Song, R.; Wang, X.; Wang, Y. A survey on fairness in large language models. arXiv 2023, arXiv:2308.10149. — [arXiv:2308.10149](https://arxiv.org/abs/2308.10149) find
  75. Chang, Y.; Wang, X.; Wang, J.; Wu, Y.; Yang, L.; Zhu, K.; Chen, H.; Yi, X.; Wang, C.; Wang, Y.; et al. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol. 2024, 15, 1–45.
  76. Vulic, I.; Moens, M.F. Cross-lingual semantic similarity of words as the similarity of their semantic word responses. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2013), Atlanta, GA, USA,… find
  77. Hendricks, L.A.; Burns, K.; Saenko, K.; Darrell, T.; Rohrbach, A. Women also snowboard: Overcoming bias in captioning models. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 771–787. find
  78. Lee, K.; He, L.; Lewis, M.; Zettlemoyer, L. End-to-end neural coreference resolution. arXiv 2017, arXiv:1707.07045.
  79. Rudinger, R.; Naradowsky, J.; Leonard, B.; Van Durme, B. Gender bias in coreference resolution. arXiv 2018, arXiv:1804.09301.
  80. Hovy, D.; Yang, D. The importance of modeling social factors of language: Theory and practice. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, 6–11 June 2021; pp. 588–602.
  81. Cer, D.; Diab, M.; Agirre, E.; Lopez-Gazpio, I.; Specia, L. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv 2017, arXiv:1708.00055.
  82. Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; Chang, K.W. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv 2017, arXiv:1707.09457.
  83. Diaz, F.; Mitra, B.; Craswell, N. Query expansion with locally-trained word embeddings. arXiv 2016, arXiv:1605.07891. find
  84. Tan, L.; Bond, F. Building and annotating the linguistically diverse NTU-MC (NTU-multilingual corpus). In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, Singapore, 16–18 December 2011; pp. 362–371. find
  85. Bowman, S.R.; Angeli, G.; Potts, C.; Manning, C.D. A large annotated corpus for learning natural language inference. arXiv 2015, arXiv:1508.05326.
  86. Chen, Q.; Zhu, X.; Ling, Z.; Wei, S.; Jiang, H.; Inkpen, D. Enhanced LSTM for natural language inference. arXiv 2016, arXiv:1609.06038. find
  87. De-Arteaga, M.; Romanov, A.; Wallach, H.; Chayes, J.; Borgs, C.; Chouldechova, A.; Geyik, S.; Kenthapadi, K.; Kalai, A.T. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA,… find
  88. Meng, Y.; Zhang, Y.; Huang, J.; Xiong, C.; Ji, H.; Zhang, C.; Han, J. Text classification using label names only: A language model self-training approach. arXiv 2020, arXiv:2010.07245.
  89. Díaz, M.; Johnson, I.; Lazar, A.; Piper, A.M.; Gergle, D. Addressing age-related bias in sentiment analysis. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montréal, QC, Canada, 21–26 April 2018; pp. 1–14. find
  90. Rajpurkar, P.; Jia, R.; Liang, P. Know what you don’t know: Unanswerable questions for SQuAD. arXiv 2018, arXiv:1806.03822.
  91. Webster, K.; Recasens, M.; Axelrod, V.; Baldridge, J. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Trans. Assoc. Comput. Linguist. 2018, 6, 605–617.
  92. Bender, E.M.; Friedman, B. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Linguist. 2018, 6, 587–604. find
  93. Pang, B.; Lee, L. Opinion mining and sentiment analysis. Found. Trends® Inf. Retr. 2008, 2, 1–135. find
  94. Misiunas, K.; Keyser, U.F. Density-dependent speed-up of particle transport in channels. Phys. find
  95. Burnap, P.; Williams, M.L. Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy Internet 2015, 7, 223–242. find
  96. Hovy, D.; Søgaard, A. Tagging performance correlates with author age. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Beijing, China, 26–31 July 2015, pp. 48…
  97. Blodgett, S.L.; O’Connor, B. Racial disparity in natural language processing: A case study of social media african-american english. arXiv 2017, arXiv:1707.00061.
  98. Rajpurkar, P.; Zhang, J.; Lopyrev, K.; Liang, P. Squad: 100,000+ questions for machine comprehension of text. arXiv 2016, arXiv:1606.05250.
  99. Gardner, M.; Grus, J.; Neumann, M.; Tafjord, O.; Dasigi, P.; Liu, N.; Peters, M.; Schmitz, M.; Zettlemoyer, L. Allennlp: A deep semantic natural language processing platform. arXiv 2018, arXiv:1803.07640.
  100. Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; et al. Natural questions: A benchmark for question answering research. Trans. Assoc. Comput. Linguist. 2019, 7, 453–466.
  101. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I. Language models are unsupervised multitask learners. OpenAI Blog 2019, 1, 9. find
  102. Birhane, A.; Prabhu, V.U. Large image datasets: A pyrrhic win for computer vision? In Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2021; pp. 1536–1546. find
  103. Zhang, Y.; Sun, S.; Galley, M.; Chen, Y.C.; Brockett, C.; Gao, X.; Gao, J.; Liu, J.; Dolan, B. Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv 2019, arXiv:1911.00536.
  104. Hitsch, G.J.; Hortaçsu, A.; Ariely, D. Matching and sorting in online dating. Am. Econ. Rev. 2010, 100, 130–163. find
  105. Wagner, C.; Graells-Garrido, E.; Garcia, D.; Menczer, F. Women through the glass ceiling: Gender asymmetries in Wikipedia. EPJ Data Sci. 2016, 5, 5. find
  106. Hovy, D.; Spruit, S.L. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Berlin, Germany, 7–12 August 2016; pp. 591–598.
  107. Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J.W.; Wallach, H.; Iii, H.D.; Crawford, K. Datasheets for datasets. Commun. ACM 2021, 64, 86–92. find
  108. Zhang, S.; Yao, L.; Sun, A.; Tay, Y. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. (CSUR) 2019, 52, 1–38. find
  109. Ekstrand, M.D.; Tian, M.; Azpiazu, I.M.; Ekstrand, J.D.; Anuyah, O.; McNeill, D.; Pera, M.S. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Proceedings of the Conference on Fairness, Accountability and Transparency, PMLR, New… find
  110. Chen, R.C.; Ai, Q.; Jayasinghe, G.; Croft, W.B. Correcting for recency bias in job recommendation. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China, 3–7 November 2019; pp. 2185–2188. find
  111. Biega, A.J.; Gummadi, K.P.; Weikum, G. Equity of attention: Amortizing individual fairness in rankings. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA, 8–12 July 2018; pp. 405–414. find
  112. Burke, R.; Sonboli, N.; Ordonez-Gauger, A. Balanced neighborhoods for multi-sided fairness in recommendation. In Proceedings of the Conference on Fairness, Accountability and Transparency, PMLR, New York, NY, USA, 23–24 February 2018; pp. 202–214. find
  113. Karimi, M.; Jannach, D.; Jugovac, M. News recommender systems–Survey and roads ahead. Inf. find
  114. Lakew, S.M.; Federico, M.; Negri, M.; Turchi, M. Multilingual neural machine translation for low-resource languages. IJCoL Ital. J. Comput. Linguist. 2018, 4, 11–25. find
  115. Vaswani, A. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 5998–6008.
  116. Prates, M.O.; Avelar, P.H.; Lamb, L.C. Assessing gender bias in machine translation: A case study with google translate. Neural Comput. Appl. 2020, 32, 6363–6381.
  117. Neri, D.; Soldani, J.; Zimmermann, O.; Brogi, A. Design principles, architectural smells and refactorings for microservices: A multivocal review. SICS Softw.-Intensive-Cyber-Phys. Syst. 2020, 35, 3–15. find
  118. Vanmassenhove, E.; Shterionov, D.; Way, A. Lost in translation: Loss and decay of linguistic richness in machine translation. arXiv 2019, arXiv:1906.12068.
  119. Sennrich, R. Neural Machine Translation; Institute for Language, Cognition and Computation University of Edinburgh: Edinburgh, UK, 2016; Volume 18. find
  120. Otterbacher, J.; Bates, J.; Clough, P. Competent men and warm women: Gender stereotypes and backlash in image search results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; pp. 6620–6631. find
  121. Shen, T.; Lei, T.; Barzilay, R.; Jaakkola, T. Style transfer from non-parallel text by cross-alignment. Adv. Neural Inf. Process. Syst. 2017, 30, 6833–6844.
  122. Perez-Beltrachini, L.; Lapata, M. Models and datasets for cross-lingual summarisation. arXiv 2022, arXiv:2202.09583. find
  123. Liang, P.P.; Wu, C.; Morency, L.P.; Salakhutdinov, R. Towards understanding and mitigating social biases in language models. In Proceedings of the International Conference on Machine Learning, PMLR, Online, 18–24 July 2021; pp. 6565–6576.
  124. Simmons, G.; Hare, C. Large language models as subpopulation representative models: A review. arXiv 2023, arXiv:2310.17888. — [arXiv:2310.17888](https://arxiv.org/abs/2310.17888) find
  125. Wang, A.; Morgenstern, J.; Dickerson, J.P. Large language models cannot replace human participants because they cannot portray identity groups. arXiv 2024, arXiv:2402.01908. — [arXiv:2402.01908](https://arxiv.org/abs/2402.01908) find
  126. Gorti, A.; Gaur, M.; Chadha, A. Unboxing Occupational Bias: Grounded Debiasing LLMs with US Labor Data. arXiv 2024, arXiv:2408.11247. — [arXiv:2408.11247](https://arxiv.org/abs/2408.11247) find
  127. Rozado, D. The political preferences of LLMs. PLoS ONE 2024, 19, e0306621. find
  128. Khanuja, S.; Ruder, S.; Talukdar, P. Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages. arXiv 2022, arXiv:2205.12676. find
  129. Csáky, R.; Purgai, P.; Recski, G. Improving neural conversational models with entropy-based data filtering. arXiv 2019, arXiv:1905.05471. find
  130. Durward, M.; Thomson, C. Evaluating Vocabulary Usage in LLMs. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), Mexico City, Mexico, 20 June 2024; pp. 266–282. de Vassimon Manela, D.; Errington, D.; Fisher, T.; van Breugel, B.; Minervini,… find
  131. Dang, V.M.H.; Verma, R.M. Data quality in NLP: Metrics and a comprehensive taxonomy. In Proceedings of the International Symposium on Intelligent Data Analysis, Stockholm, Sweden, 24–26 April 2024; Springer: Berlin/Heidelberg, Germany, 2024; pp. 217–229. find
  132. Shen, Z.; Tao, T.; Ma, L.; Neiswanger, W.; Hestness, J.; Vassilieva, N.; Soboleva, D.; Xing, find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guo-2024-bias-large-language-models-origin/