Elamur Research⌕
☰
home / papers / 2025 / Amharic News Topic Classification: Dataset and Transformer-Based Model Benchmarks

Amharic News Topic Classification: Dataset and Transformer-Based Model Benchmarks

Paper · 2025 · Dagnachew Mekonnen Marilign and Eyob Nigussie Alemu

Summary

Amharic is widely used but has few large labeled datasets for news topic classification. This paper addresses that gap with a bigger corpus and a set of baseline models. The authors scraped 12 Amharic news outlets with BeautifulSoup and kept each outlet's own editorial category as the label. They merged 92,792 new articles with a manually cleaned version of the earlier Azime and Mohammed (2021) collection, giving 144,201 articles across six topics (2011 to late 2024). Five transformers were fine-tuned for five epochs on a stratified 70/10/20 split: mBERT, XLM-R, DistilBERT, AfriBERTa and AfroXLMR. AfriBERTa and XLM-R scored best, with weighted F1 of about 0.90 and macro F1 of 0.89 and 0.88. The general-purpose mBERT and DistilBERT stayed near 0.57 macro F1. Moving from the original data to the expanded data improved most models, especially on minority classes. The resource and benchmarks give Amharic NLP a reproducible baseline. The results also suggest that Africa-centric or large multilingual pretraining works better than generic small models for this language.

Key points

  • Corpus size and sources: The expanded corpus has 144,201 articles from 12 outlets, up from 51,409 after cleaning (+180%).
  • Classes: Six labels: Local, Entertainment, Sport, Business, International, Politics. The data is heavily imbalanced: Local has ~63k articles and Entertainment only ~1.1k.
  • Record fields: Each record holds headline, body, label and source URL. Publication date and view count are included when available, otherwise marked NA.
  • Training setup: Hugging Face Trainer with lr 5e-5, weight decay 0.1, fp16, gradient accumulation 4, batch 16 or 32, and a 512-token limit. Training ran on a Kaggle P100 GPU. The input is the headline joined with the body.
  • Macro F1, original → expanded:
    • AfriBERTa: 0.87 → 0.89
    • XLM-R: 0.85 → 0.88
    • AfroXLMR: 0.84 → 0.88
    • mBERT: 0.50 → 0.57
    • DistilBERT: 0.60 → 0.57 (the only model that dropped)
    • Entertainment stayed the hardest class.

Interesting ideas

  • I1. Choosing macro F1 as the main metric under class imbalance keeps big classes from hiding failures on rare ones; a gap between weighted and macro scores (e.g. mBERT 0.63 vs 0.57) is itself a diagnostic.
  • I2. Benchmarking every model on both the old and the enlarged dataset, with the same protocol, isolates what added data buys per class and reveals models that do not benefit (here DistilBERT).
  • I3. Reusing labels that publishers already assign gives cheap large-scale supervision; the authors pair this with a manual cleanup pass and flag possible editorial bias, suggesting a human-checked gold subset.
  • I4. Storing provenance and optional metadata (URL, date, views, NA when missing) with every record keeps the corpus usable for later tasks such as temporal analysis.
  • I5. Combining a per-class F1 table with a confusion matrix shows that small, semantically overlapping categories stay hard even with more data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Report macro-averaged scores across dependency relations or frame types next to the overall figure, so rare constructions cannot be masked by frequent ones.
  • I2 (learn): When the silver corpus grows, retrain the parsers on old and new versions under one fixed evaluation, to show which models and labels actually gain.
  • I3 (learn): Treat externally given labels as one more witness in silver agreement, and keep a small human-verified gold slice to measure how biased that witness is.
  • I4 (learn): Tag each silver sentence with its source and date, so errors can be traced back to a particular source and filtered.
  • I5 (answer): Break down reader accuracy by question type with a confusion view, to see which kinds of school-test or fable questions get mixed up.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Addis Ababa UniversityAddis Ababa · Ethiopia17 works · 77 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksEvaluation & benchmarks · 1,341 works

Cite

Cite as Dagnachew Mekonnen Marilign and Eyob Nigussie Alemu. 2025. Amharic News Topic Classification: Dataset and Transformer-Based Model Benchmarks. Proceedings of the 9th Widening NLP Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.winlp-main.23

@inproceedings{marilign2025amharic,
  title = {{Amharic News Topic Classification: Dataset and Transformer-Based Model Benchmarks}},
  author = {Dagnachew Mekonnen Marilign and Eyob Nigussie Alemu},
  booktitle = {Proceedings of the 9th Widening NLP Workshop},
  year = {2025},
  doi = {10.18653/v1/2025.winlp-main.23},
  url = {https://doi.org/10.18653/v1/2025.winlp-main.23}
}

References

18 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, and 42 others. 2021. MasakhaNER: Named entity recognition for African languages. Transactions of the Association for Computational Linguistics, 9:1116–1131.
  3. David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Alabi, Atnafu Lambebo Tonja, Christine Mwase, Odunayo Ogundepo, Bonaventure F. P. Dossou, Akintunde Oladipo, Doreen Nixdorf, Chris Chinenye Emezue, Sana Al-azzawi, Blessing Sibanda, Davis David, Lolwethu Ndolela, Jonathan Mukiibi, Tunde Ajayi, Tatiana Moteu, Brian Odhiambo, and 46 others. 2023. MasakhaNEWS: News topic classification for African languages. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 144–159, Nusa Dua, Bali. Association for Computational Linguistics.
  4. David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba O. Alabi, Shamsuddeen H. Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, and 26 others. 2022. MasakhaNER 2.0: Africa-centric transfer learning for named entity recognition. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4488–4508, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  5. Jesujoba Oluwadara Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022. Multilingual language model adaptive fine-tuning: A study on african languages. In 3rd Workshop on African Natural Language Processing. find
  6. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics. ↗
  7. Ali Saleh Alammary. 2022. Bert models for arabic text classification: a systematic review. Applied Sciences, 12(11):5720. find
  8. Israel Abebe Azime and Nebil Mohammed. 2021. An amharic news text classification dataset. arXiv preprint arXiv:2103.05639. ↗
  9. V Sanh. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. ↗
  10. Avihay Chriqui and Inbal Yahav. 2022. Hebert and hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science, 1(1):81–95. ↗
  11. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  13. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. 2021. Alephbert: A hebrew large pre-trained language model to start-off your hebrew nlp application with. arXiv preprint arXiv:2104.04052. ↗
  14. Atnafu Lambebo Tonja, Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Moges Ahmed Mehamed, Olga Kolesnikova, and Seid Muhie Yimam. 2023. Natural language processing in Ethiopian languages: Current state, challenges, and opportunities. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 126–139, Dubrovnik, Croatia. Association for Computational Linguistics. find
  15. Demeke Endalie and Getamesay Haile. 2021. Automated amharic news categorization using deep learning models. Computational Intelligence and Neuroscience, 2021(1):3774607. find
  16. Worku Kelemework. 2013. Automatic amharic text news classification: Aneural networks approach. Ethiopian Journal of Science and Technology, 6(2):127–137. find
  17. Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, Nedjma Ousidhoum, David Ifeoluwa Adelani, Seid Muhie Yimam, Ibrahim Sa’id Ahmad, Meriem Beloucif, Saif M. Mohammad, Sebastian Ruder, Oumaima Hourrane, Pavel Brazdil, Alipio Jorge, Felermino Dário Mário António Ali, Davis David, Salomey Osei, Bello Shehu Bello, Falalu Ibrahim, Tajuddeen Gwadabe, and 8 others. 2023. AfriSenti: A Twitter sentiment analysis benchmark for African languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13968–13981, Singapore. Association for Computational Linguistics.
  18. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021. Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marilign-2025-amharic-news-topic-classification-dataset/