MasakhaNER: Named Entity Recognition for African Languages
Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2021.tacl-1.66.pdf
Summary
African languages are largely missing from NLP resources, and NER in particular has had almost no public, human-annotated benchmarks for them. A large community effort coordinated through Masakhane, which brought together native speakers, annotators, and NLP researchers, built MasakhaNER. It is the first sizable, high-quality, openly released NER dataset for ten widely spoken African languages: Amharic, Hausa, Igbo, Kinyarwanda, Luganda, Luo, Nigerian-Pidgin, Swahili, Wolof, and Yorùbá. The texts come from local news sources, so the entities reflect what matters to speakers of each language. The authors describe the typology and orthography of each language, including scripts, tone, and affixation. They then benchmark a CNN-BiLSTM-CRF baseline and fine-tuned multilingual transformers (mBERT, XLM-R) in supervised, cross-domain, cross-lingual, and regional multi-language settings. Key findings:
- Transfer from geographically and linguistically close African languages beats transfer from English.
- Pooling training data across languages gives modest gains.
- Unseen and multi-word entities remain hard.
The data, code, and models are public, which gives African NLP a reproducible starting point.
Key points
- Entity types annotated are PER, LOC, ORG, and DATE. The languages span Afro-Asiatic, Niger-Congo, Nilo-Saharan, and an English Creole, with speaker counts ranging from about 4M (Luo) to 98M (Swahili).
- Zero-shot cross-lingual transfer with XLM-R-base:
- A model trained on only about 2K Nigerian-Pidgin sentences performs about as well as one trained on eng-CoNLL03 (about 14K sentences).
- Hausa as the source language gives the best average F1, more than 16 points above eng-CoNLL03.
- Cross-domain transfer from WikiAnn (Wikipedia) to news is weak. Zero-shot F1 stays below 40 for all five languages tested, and below 10 for Kinyarwanda and Yorùbá.
- Training one model on all nine non-Amharic languages helps every language except Swahili. Across ibo, kin, lug, luo, wol, and yor, the average gain is 1.6 F1.
- Error analysis shows two main weak spots:
- Entities never seen in training lose about 5 F1 points.
- Entities of four or more words lose about 20 F1 points.
Cited by
- Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning
- A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios
- Challenges and Strategies in Cross-Cultural NLP
- Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya
- Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages
- Lifting the Curse of Multilinguality by Pre-training Modular Transformers
- Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
- Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
- Ethical and social risks of harm from Language Models
Cite
@article{adelani2021masakhaner,
title = {{MasakhaNER: Named Entity Recognition for African Languages}},
author = {Adelani, David Ifeoluwa and Abbott, Jade and Neubig, Graham and D'souza, Daniel and Kreutzer, Julia and Lignos, Constantine and Palen-Michel, Chester and Buzaaba, Happy and Rijhwani, Shruti and Ruder, Sebastian and Mayhew, Stephen and Azime, Israel Abebe and Muhammad, Shamsuddeen H. and Emezue, Chris Chinenye and Nakatumba-Nabende, Joyce and Ogayo, Perez and Anuoluwapo, Aremu and Gitau, Catherine and Mbaye, Derguene and Alabi, Jesujoba and Yimam, Seid Muhie and Gwadabe, Tajuddeen Rabiu and Ezeani, Ignatius and Niyongabo, Rubungo Andre and Mukiibi, Jonathan and Otiende, Verrah and Orife, Iroro and David, Davis and Ngom, Samba and Adewumi, Tosin and Rayson, Paul and Adeyemi, Mofetoluwa and Muriuki, Gerald and Anebi, Emmanuel and Chukwuneke, Chiamaka and Odu, Nkiruka and Wairagala, Eric Peter and Oyerinde, Samuel and Siro, Clemencia and Bateesa, Tobius Saul and Oloyede, Temilola and Wambui, Yvonne and Akinode, Victor and Nabagereka, Deborah and Katusiime, Maurice and Awokoya, Ayodele and MBOUP, Mouhamadane and Gebreyohannes, Dibora and Tilaye, Henok and Nwaike, Kelechi and Wolde, Degaga and Faye, Abdoulaye and Sibanda, Blessing and Ahia, Orevaoghene and Dossou, Bonaventure F. P. and Ogueji, Kelechi and DIOP, Thierno Ibrahima and Diallo, Abdoulaye and Akinfaderin, Adewale and Marengereke, Tendai and Osei, Salomey},
journal = {Transactions of the Association for Computational Linguistics},
year = {2021},
doi = {10.1162/tacl\_a\_00416},
url = {https://doi.org/10.1162/tacl_a_00416}
}References
- D. Adelani, Dana Ruiter, J. Alabi, Damilola Adebonojo, Adesina Ayeni, Mofetoluwa Adeyemi, Ayodele Awokoya, and C. España-Bonet. 2021. MENYO-20k: A multi-domain english-yorùbá corpus for machine translation and domain adaptation. ArXiv, abs/2103.08647. — [arXiv:2103.08647](https://arxiv.org/abs/2103.08647) find
- Željko Agić and Ivan Vulić. 2019. JW300: A wide-coverage parallel corpus for low-resource languages. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3204–3210, Florence, Italy. Association for Computational Linguistics. find our card