Elamur Research⌕
☰
home / papers / 2026 / Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens

Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens

Summary

Most work on low-resourced languages tries to gather ever-larger corpora, but few studies check what those corpora actually contain. This paper audits machine translation data for Afan Oromo, Amharic and Tigrinya, looking at domain coverage and at how gender is represented. The authors surveyed 17 dataset papers. They then examined the training corpus NLLB and three evaluation benchmarks: FLORES, HornMT and MAFAND. Their methods combined automated and human analysis: NER on person names, morphological analysis of verb gender, LDA topic modeling on sentences split into feminine and masculine groups, and qualitative review of sampled text. They found that the domains of training and test data do not match. Training text leans toward religion and politics, while benchmarks favour news, health and sports. All datasets strongly favour men, through names, verb forms and stereotyped occupations. Toxic depictions of women were most visible for the language with the most data. The work shows that data volume does not guarantee quality. It argues that bias checks should be built into dataset design rather than applied after models are trained.

Key points

  • Of the 17 dataset papers surveyed, over 93% of the datasets include religious text.
  • NLLB parallel data sizes are 16.14M sentence pairs for Amharic, 3.23M for Afan Oromo and 1.39M for Tigrinya. FLORES and HornMT have about 2K sentences per language, and MAFAND (about 1.94K) covers Amharic only.
  • More than 80% of person names in the datasets are stereotypically male. The gap between male and female verb forms reaches up to 72 percentage points.
  • Occupations such as engineer are consistently linked to the male gender, and certain adjectives also pattern by gender.
  • The analysis covers only the target-language side of the data and not the English side, and it relied on sampling for NLLB. Code and data are available on GitHub at hhnigatu/EvaluatingMTDatasets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Topics: Machine translation

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.330

@inproceedings{nigatu2026evaluating,
  title = {{Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens}},
  author = {Nigatu, Hellina Hailu and Mamo, Bethelhem Yemane and Balcha, Bontu Fufa and Tesfaye, Debora Taye and Zewdie, Elbethel Daniel and Nesiru, Ikram Behiru and Hailu, Jitu Ewnetu and Yayo, Senait Mengesha},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.330},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.330}
}

References

84 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdo Ababor. 2021. Boosting afaan oromo named entity recognition with multiple methods. International Journal of Information Engineering & Electronic Business, 13(5). find
  2. Solomon Teferra Abate, Michael Melese, Martha Yifiru Tachbelie, Million Meshesha, Solomon Atinafu, Wondwossen Mulugeta, Yaregal Assabie, Hafte Abera, Binyam Ephrem, Tewodros Abebe, Wondimagegnhue Tsegaye, Amanuel Lemma, Tsegaye Andargie, and Seifedin Shifaw. 2018. Parallel corpora for bi-lingual English-Ethiopian languages statistical machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3102–3111, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  3. Tesfaw Abebaw Ejigu. 2024. The overlooked roles of women in the patriotic resistance movement in Bure Damot, 19361941. Cogent Arts & Humanities, 11(1):2390786. Publisher: Cogent OA _eprint: https://doi.org/10.1080/23311983.2024.2390786. ↗
  4. D. Adelani, Jade Abbott, Graham Neubig, Daniel D’Souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, S. Muhammad, Chris C. Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Anuoluwapo Aremu, Catherine Gitau, Derguene Mbaye, J. Alabi, Seid Muhie Yimam, Tajuddeen R. Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, V. Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin P. Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, C. Chukwuneke, N. Odu, Eric Peter Wairagala, S. Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane Mboup, D. Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima Diop, A. Diallo, Adewale Akinfaderin, T. Marengereke, and Salomey Osei. 2021a. Masakhaner: Named entity recognition for african languages. ArXiv, abs/2103.11811. ↗ our card
  5. David Adelani, Jesujoba Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen Muhammad, Guyo Jarso, Oreen Yousuf, Andre Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin Ajibade, Tunde Ajayi, Yvonne Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anuoluwapo Aremu, Perez Ogayo, Jonathan Mukiibi, Fatoumata Ouoba Kabore, Godson Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire Memdjokam Koagne, Edwin Munkoh-Buabeng, Valencia Wagner, Idris Abdulmumin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, and Sam Manthalu. 2022. A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3053–3070, Seattle, United States. Association for Computational Linguistics. find
  6. David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel Dsouza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei. 2021b. MasakhaNER: Named Entity Recognition for African Languages. Transactions of the Association for Computational Linguistics, 9:1116–1131. ↗ our card
  7. Sisay Adugna and Andreas Eisele. 2010. English — Oromo machine translation: An experiment using a statistical approach. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC‘10), Valletta, Malta. European Language Resources Association (ELRA). find
  8. Jesujoba Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022a. Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning. ACL Anthology, pages 4336–4349. ↗ our card
  9. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022b. Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. ↗ our card
  10. Jisun An, Haewoon Kwak, and Yong-Yeol Ahn. 2018. SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2450–2461, Melbourne, Australia. Association for Computational Linguistics. find
  11. Yeabsira Asefa Ashengo, Rosa Tsegaye Aga, and Surafel Lemma Abebe. 2021. Context based machine translation with recurrent neural network for englishamharic translation. Machine Translation, 35:19 – 36. find
  12. Gashaw Ayanaw and Abebe Alewond. 2024. Mainstreaming gender equality: A comparative analysis on gender roles of the Awura Amba and the nearby community, Amhara Regional State, Ethiopia. Social Sciences & Humanities Open, 10:101149. ↗
  13. Selam Balehey and Mulubrhan Balehegn. 2019. The Art, Aesthetics and Gender Significance of Ashenda girls’ Festival in Tigray, Northern Ethiopia. find
  14. Adugna Barkessa. 2020. A Critical Discourse Analysis of Gendered Representations in Afaan Oromoo Textbook. East African Journal of Social Sciences and Humanities, 5(1):19–36. find
  15. Martijn Bartelds, Nay San, Bradley McDonnell, Dan Jurafsky, and Martijn Wieling. 2023. Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 715–729, Toronto, Canada. Association for Computational Linguistics. find
  16. Tadesse Destaw Belay, Atnafu Lambebo Tonja, Olga Kolesnikova, Seid Muhie Yimam, Abinew Ali Ayele, Silesh Bogale Haile, Grigori Sidorov, and Alexander Gelbukh. 2022. The effect of normalization for bi-directional amharic-english neural machine translation. Preprint, arXiv:2210.15224. ↗
  17. Emily Bender. 2019. The #BenderRule: On Naming the Languages We Study and Why It Matters. find
  18. Zemicheal Berihu, Gebremariam Mesfin, Mulugeta At-sibaha, and Tor-Morten Grønli. 2020. Enhancing bi-directional english-tigrigna machine translation using hybrid approach. find
  19. Yohanens Biadgligne and Kamel Smaïli. 2021. Parallel corpora preparation for english-amharic machine translation. In Advances in Computational Intelligence, pages 443–455, Cham. Springer International Publishing. find
  20. Yohannes Biadgligne and Kamel Smaili. 2022. Offline corpus augmentation for english-amharic machine translation. In 2022 5th International Conference on Information and Computer Technologies (ICICT), pages 128–135. find
  21. Abeba Birhane, Vinay Prabhu, Sang Han, and Vishnu Naresh Boddeti. 2023a. On Hate Scaling Laws For Data-Swamps. arXiv preprint. ArXiv:2306.13141 [cs]. ↗
  22. Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti, and Alexandra Sasha Luccioni. 2023b. Into the LAIONs Den: Investigating Hate in Multimodal Datasets. find
  23. Belete Bizuneh. 2001. Women in Ethiopian History: A Bibliographic Review. Northeast African Studies, 8(3):7–32. Publisher: Michigan State University Press. ↗
  24. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res., 3(null):993–1022. ↗ our card
  25. Su Lin Blodgett, Solon Barocas, Hal Daumé Iii, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of Bias in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Online. Association for Computational Linguistics. ↗ our card
  26. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1004–1015, Online. Association for Computational Linguistics. ↗
  27. Sisay Adugna Chala, Bekele Debisa, Amante Diriba, Silas Getachew, Chala Getu, and Solomon Shiferaw. 2021. Crowdsourcing parallel corpus for english-oromo neural machine translation using community engagement platform. ArXiv, abs/2102.07539. ↗
  28. Tyler A. Chang, Catherine Arnett, Zhuowen Tu, and Benjamin K. Bergen. 2024. Goldfish: Monolingual Language Models for 350 Languages. arXiv preprint. ArXiv:2408.10441 [cs]. ↗
  29. Pieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett, and Zeerak Talat. 2024. Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21669–21691, Miami, Florida, USA. Association for Computational Linguistics. find
  30. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. find
  31. Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Emezue. 2022. AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages. In Proceedings of the Third Workshop on Simple and Efficient Natural Language Processing (SustaiNLP), pages 52–64, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. ↗
  32. Mbonu Chinedu Emmanuel, Oreen Yousuf, Kaosarat Aina, and Davis David. 2022. The African Stopwords project: curating stopwords for African languages. AfricaNLP. find
  33. Fitsum Gaim, Wonsuk Yang, and Jong C. Park. 2022. GeezSwitch: Language Identification in Typologically Related Low-resourced East African Languages. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6578–6584, Marseille, France. European Language Resources Association. find
  34. Michael Gasser. 2011. HornMorpho: a system for morphological processing of Amharic, Oromo, and Tigrinya. ResearchGate. find
  35. Andargachew Mekonnen Gezmu, Andreas Nürnberger, and Tesfaye Bayu Bati. 2022. Extended parallel corpus for Amharic-English machine translation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6644–6653, Marseille, France. European Language Resources Association. find
  36. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzman, and Angela Fan. 2021. The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation. arXiv preprint. ArXiv:2106.03193 [cs]. ↗
  37. Rishav Hada, Safiya Husain, Varun Gumma, Harshita Diddee, Aditya Yadavalli, Agrima Seth, Nidhi Kulkarni, Ujwal Gadiraju, Aditya Vashistha, Vivek Seshadri, and Kalika Bali. 2024. Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, pages 1926–1939, New York, NY, USA. Association for Computing Machinery. find
  38. Asmelash Teka Hadgu, Gebrekirstos G. Gebremeskel, and Abel Aregawi. 2022. Machine translation benchmark dataset for languages in the horn of africa. find
  39. William Held, Camille Harris, Michael Best, and Diyi Yang. 2023. A Material Lens on Coloniality in NLP. arXiv preprint. ArXiv:2311.08391 [cs]. ↗
  40. Endalkachew Lelisa Duressa. 2018. Review on the Historical Prospective of Siiqqee: The Ancient African Indigenous Institution for Empowering Women. Journal of Philosophy, Culture and Religion, 36. find
  41. Ben Hutchinson. 2024. Modeling the Sacred: Considerations when Using Religious Texts in Natural Language Processing. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1029–1043, Mexico City, Mexico. Association for Computational Linguistics. ↗
  42. Chris Emezue, Hellina Nigatu, Cynthia Thinwa, Helper Zhou, Shamsuddeen Muhammad, Lerato Louis, Idris Abdulmumin, Samuel Oyerinde, Benjamin Ajibade, Olanrewaju Samuel, Oviawe Joshua, Emeka Onwuegbuzia, Handel Emezue, Ifeoluwatayo A. Ige, Atnafu Lambebo Tonja, Chiamaka Chukwuneke, Bonaventure F. P. Dossou, Naome A. Etori, Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics. find our card
  43. Raviraj Joshi, Kanishk Singla, Anusha Kamath, Raunak Kalani, Rakesh Paul, Utkarsh Vaidya, Sanjay Singh Chauhan, Niranjan Wartikar, and Eileen Long. 2025. Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus. arXiv preprint. ArXiv:2410.14815 [cs]. ↗
  44. Hannah Kirk, Abeba Birhane, Bertie Vidgen, and Leon Derczynski. 2022. Handling and Presenting Harmful Text in NLP Research. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 497–510, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  45. Tsegay Kiros. 2020. Statistical machine translator for english to tigrigna translation. find
  46. Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ball, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. 2022. Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets. Transactions of the Association for Computational Linguistics, 10:50–72. ↗ our card
  47. Zelalem Leyew. 2003. Amharic Personal Nomenclature: A Grammar and Sociolinguistic Insight. ↗
  48. Li Lucy, Camilla Griffiths, Sarah Levine, Jennifer L. Eberhardt, Dorottya Demszky, and David Bamman. 2025. Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes. ACL Anthology, pages 22585–22610. find
  49. Meseret Mamo. 2016. Ye-Kake Wurdowet. find
  50. Andrew Kachites McCallum. 2002. Mallet: MAchine Learning for LanguagE Toolkit. find
  51. Nikita Mehandru, Samantha Robertson, and Niloufar Salehi. 2022. Reliable and Safe Use of Machine Translation in Medical Settings. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, pages 2016–2025, New York, NY, USA. Association for Computing Machinery. find
  52. Tsegaye Mekonnen. 2018. Amharic English Machine Translation Corpus prepared through website crawelling and custom preprocessing. find
  53. Tesfaye Mesele and Yohannes Asfaw. 2019. Critical Discourse Analysis of Gender Conceptions in Preparatory Tigrigna Language Textbooks. Journal of Educational and Behavioral Sciences, 2(1). find
  54. Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, and Zeerak Talat. 2025. SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11995–12041, Albuquerque, New Mexico. Association for Computational Linguistics. find
  55. Abebech Gutema Murra. 2023. The prevalence of gender bias in early childhood English and Amharic language textbooks. In Early Childhood Language Education and Literacy Practices in Ethiopia. Routledge. Num Pages: 14. ↗
  56. Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa Berhe, Mofetoluwa Adeyemi, Masabata Mokgesi-Selinga, Lawrence Okegbemi, Laura Martinus, Kolawole Tajudeen, Kevin Degila, Kelechi Ogueji, Kathleen Siminyu, Julia Kreutzer, Jason Webster, Jamiil Toure Ali, Jade Abbott, Iroro Orife, Ignatius Ezeani, Idris Abdulkadir Dangana, Herman Kamper, Hady Elsahar, Goodness Duru, Ghollah Kioko, Murhabazi Espoir, Elan van Biljon, Daniel Whitenack, Christopher Onyefuluchi, Chris Chinenye Emezue, Bonaventure F. P. Dossou, Blessing Sibanda, Blessing Bassey, Ayodele Olabiyi, Arshath Ramkilowan, Alp Öktem, Adewale Akinfaderin, and Abdallah Bashir. 2020. Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2144–2160, Online. Association for Computational Linguistics. find our card
  57. Team NLLB, Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Jeff Wang, and Meta Ai. 2022. No Language Left Behind: Scaling Human-Centered Machine Translation. ↗
  58. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021a. Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  59. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021b. Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages. ACL Anthology, pages 116–126. find our card
  60. Tolulope Ogunremi, Dan Jurafsky, and Christopher Manning. 2023. Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1251–1266, Dubrovnik, Croatia. Association for Computational Linguistics. find
  61. Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021. Gender Bias in Machine Translation. Transactions of the Association for Computational Linguistics, 9:845–874. ↗ our card
  62. Beatrice Savoldi, Sara Papi, Matteo Negri, Ana Guerberof-Arenas, and Luisa Bentivogli. 2024. What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18048–18076, Miami, Florida, USA. Association for Computational Linguistics. find our card
  63. Walelign Tewabe Sewunetie, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Hellina Hailu Nigatu, Gashaw Kidanu, Zewdie Mossie, Hussien Seid, and Seid Muhie Yimam. 2024. Gender Bias Evaluation in Machine Translation for Amharic, Tigrinya, and Afaan Oromoo. find
  64. Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David I. Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Wei-Yin Ko, Sebastian Ruder, Madeline Smith, Antoine Bosselut, Alice Oh, Andre F. T. Martins, Leshem Choshen, Daphne Ippolito, Enzo Ferrante, Marzieh Fadaee, Beyza Ermis, and Sara Hooker. 2025. Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation. arXiv preprint. ArXiv:2412.03304 [cs]. ↗ our card
  65. Oleg Sobchuk and Artjoms Šel , a. 2024. Computational thematics: comparing algorithms for clustering the genres of literary fiction. Humanities and Social Sciences Communications, 11(1). ↗
  66. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: an open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pages 4444–4451, San Francisco, California, USA. AAAI Press. ↗
  67. Dagmar Stahlberg, Friederike Braun, Lisa Irmen, and Sabine Sczesny. 2007. Representation of the Sexes in Language. In Social communication, Frontiers of social psychology, pages 163–187. Psychology Press, New York, NY, US. find
  68. Harini Suresh and John Guttag. 2021. A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. In Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO ’21, pages 1–9, New York, NY, USA. Association for Computing Machinery. find
  69. Zeerak Talat, Aurélie Névéol, Stella Biderman, Miruna Clinciu, Manan Dey, Shayne Longpre, Sasha Luccioni, Maraim Masoud, Margaret Mitchell, Dragomir Radev, Shanya Sharma, Arjun Subramonian, Jaesung Tae, Samson Tan, Deepak Tunuguntla, and Oskar Van Der Wal. 2022. You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings. In Proceedings of BigScience Episode #5 – Workshop on Challenges & Perspectives in Creating Large Language Models, pages 26–41, virtual+Dublin. Association for Computational Linguistics. find
  70. William Tan and Kevin Zhu. 2024. Nusamt-7b: Machine translation for low-resource indonesian languages with large language models. Preprint, arXiv:2410.07830. NLLB Team, Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Jeff Wang, and Meta Ai. No Language Left Behind: Scaling Human-Centered Machine Translation. ↗
  71. Yemane Tedla and Kazuhide Yamamoto. 2016. The effect of shallow segmentation on english-tigrinya statistical machine translation. In 2016 International Conference on Asian Language Processing (IALP), pages 79–82. find
  72. Yemane Keleta Tedla and Kazuhide Yamamoto. 2018. Morphological segmentation for english-to-tigrinya statistical machine translation. find
  73. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find
  74. Haoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang, Akiko Eriguchi, and Huda Khayrallah. 2025. X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale. ICLR. ↗
  75. Laure Thompson and David Mimno. 2018. Authorless Topic Models: Biasing Models Away from Known Structure. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3903–3914, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  76. Amanuel Raga Yadate. 2015. Comparative analyses of linguistic sexism in Afan Oromo, Amharic, and Gamo. ↗
  77. Jennifer Tracey and Stephanie Strassel. 2020. Basic language resources for 31 languages (plus English): The LORELEI representative and incident language packs. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), pages 277–284, Marseille, France. European Language Resources association. find
  78. Hailemariam Mehari Yohannes and Toshiyuki Amagasa. 2022. A method of named entity recognition for tigrinya. ACM SIGAPP Applied Computing Review, 22(3):56–68. ↗
  79. Lucas Nunes Vieira, Minako OHagan, and Carol OSullivan. 2021. Understanding the societal impacts of machine translation: a critical review of the literature on medical and legal use cases. Information, Communication & Society, 24(11):1515–1532. Publisher: Routledge _eprint: https://doi.org/10.1080/1369118X.2020.1776370. ↗
  80. Eric Peter Wairagala, Jonathan Mukiibi, Jeremy Francis Tusubira, Claire Babirye, Joyce Nakatumba-Nabende, Andrew Katumba, and Ivan Ssenkungu. 2022. Gender bias Evaluation in Luganda-English Machine Translation. In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 274–286, Orlando, USA. Association for Machine Translation in the Americas. find
  81. Iain Weissburg, Sathvika Anand, Sharon Levy, and Haewon Jeong. 2025. LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5650–5698, Albuquerque, New Mexico. Association for Computational Linguistics. find
  82. Michael Woldeyohannis and Million Meshesha. 2018. Experimenting statistical machine translation for ethiopic semitic languages: The case of amharic-tigrigna. find
  83. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Solomon Yitayew. 2017. Optimal alignment for bidirectional afaan oromo-english statistical machine translation. find our card
  84. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/nigatu-2026-evaluating-machine-translation-datasets-low/