Elamur Research⌕
☰
home / papers / 2025 / Supervised Machine Learning based Amharic Text Complexity Classification Using Automatic Annotator Tool

Supervised Machine Learning based Amharic Text Complexity Classification Using Automatic Annotator Tool

Paper · 2025 · Gebregziabihier Nigusie

Summary

The paper addresses readability for Amharic, a morphologically rich, low-resource Semitic language. Rare or unfamiliar vocabulary makes texts hard for low-literacy readers and second-language learners. Earlier Amharic work was limited by slow, costly manual labelling. To remove that bottleneck, the author built an automatic annotator. It splits raw text into sentences, tokenizes them, normalizes homophone spellings and extracts roots. It then marks a sentence as complex if it contains any item from a curated list of 1,113 difficult terms. Human annotators were also involved.

The resulting corpus has 20,084 sentences drawn from grade 6–12 textbooks and news. Five classifiers were compared:

  • SVM and Random Forest on Bag-of-Words features
  • LSTM and BiLSTM on Word2Vec embeddings
  • a BERT-based model

The vocabulary contains 24,148 tokens. BERT scored best at 89.4% accuracy, ahead of the recurrent models (about 84–85%) and the classical ones (about 80–84%).

The work matters as groundwork for Amharic text simplification and as a warning for downstream systems. The author observed that Google Translate mistranslated sentences containing the listed complex terms.

Key points

  • Resources: a lexicon of 1,113 complex Amharic terms, an automatic complexity annotator tool, and a corpus of 20,084 labelled sentences with a 24,148-token vocabulary.

  • Human check: in a pilot survey, three Amharic linguists marked sentences with unfamiliar words on six pages of mixed text. All three agreed on 123 sentences as complex.

  • Results:

    ModelAccuracy
    SVM83.5% in the abstract, 83.9% in the conclusion
    RF80.3%
    LSTM84.1%
    BiLSTM85.0%
    BERT89.4% (precision, recall and F1 also 89.4%)

  • Preprocessing: sentence segmentation, tokenization, root extraction, and homophone normalization, which merges different script spellings of the same sound.

  • Error analysis: misclassifications are traced to individual words that occur more often in the opposite class in the training data.

Interesting ideas

  • I1. Sentence labels can be produced automatically by matching against a curated difficulty lexicon after root extraction, so inflected forms of a listed term still match; this cheaply turns raw text into a large labelled set.
  • I2. The automatic annotator is checked against a small set where three human experts agreed unanimously, which serves as a high-confidence gold anchor for the silver data.
  • I3. Errors are explained by finding words that are more frequent in the opposite class's training data, a simple and transparent account of why a model went wrong.
  • I4. Spelling variants that sound and function alike are collapsed into one form before any lexicon lookup, so matching does not miss forms.
  • I5. The paper notes that sentences with rare lexicon terms were exactly where machine translation failed, treating complexity as a risk signal for downstream systems.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): a lemma-level lexicon match (e.g., rare WordNet senses or low-frequency lemmas) could create silver "hard sentence" labels for tagging which reader inputs need extra care.
  • I2 (learn): keep a small unanimous-agreement gold slice to audit witness-parser silver data, and report the agreement rate on it separately from bulk metrics.
  • I3 (learn): per-token class-frequency diagnostics could serve as an explainable negative-control report, showing when a learned component leans on surface words instead of structure.
  • I4 (tokens): normalize British/American and archaic spelling variants before lemma lookup in crate en, so lexicon and VerbNet matches are not lost to orthography.
  • I5 (back): flag sentences with rare or complex lemmas before generation or translation through the meta-world, and route them to a simpler paraphrase or a more careful path.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksLow-resource languages · 267 works

Cite

Cite as Gebregziabihier Nigusie. 2025. Supervised Machine Learning based Amharic Text Complexity Classification Using Automatic Annotator Tool. Proceedings of the Sixth Workshop on African Natural Language Processing (AfricaNLP 2025).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.africanlp-1.2

@inproceedings{nigusie2025supervised,
  title = {{Supervised Machine Learning based Amharic Text Complexity Classification Using Automatic Annotator Tool}},
  author = {Gebregziabihier Nigusie},
  booktitle = {Proceedings of the Sixth Workshop on African Natural Language Processing (AfricaNLP 2025)},
  year = {2025},
  doi = {10.18653/v1/2025.africanlp-1.2},
  url = {https://doi.org/10.18653/v1/2025.africanlp-1.2}
}

References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Przepiórkowski Adam and Ogrodniczuk Maciej. 2014. Advances in natural language processing. In Lecture Notes in Computer Science, pages 1–10. find
  2. J. Liu. 2017. Sentence complexity estimation for chinese-speaking learners of japanese. In Proceedings of the 31st Pacific Asia Conference on Language, Information and Computation, pages 296– 302, The National University, Philippines. find
  3. Sallab Ahmad A. Al, M. Hajj Hazem, Badaro Gilbert, R. Baly, El-Hajj Wassim, and Bashir Shaban Khaled. 2015. Deep learning models for sentiment analysis in arabic. In ANLP@ACL, pages 1–24. ↗
  4. Gasser Michael. 2011. Hornmorpho: a system for morphological processing of amharic, oromo, and tigrinya. In COMPUTEL, pages 2–5. find
  5. D. Alemu, S. Aklilu, and Y. Mengstie. 2015. Amharic teacher guide grade-9. FDRE minister of education, Addis Ababa, Ethiopia. find
  6. Gebregziabihier Nigusie and Tesfa Tegegne. 2022. Amharic text complexity classification using supervised machine learning. In Artificial Intelligence and Digitalization for Sustainable Development (ICAST 2022), pages 12–23. find
  7. G. Andrea, M. Matteo, Z. Alessandro, and A. Andrea. 2022. A survey on text classification algorithms: From text to predictions. Information, 13(83):1–39. find
  8. Gebregziabihier Nigusie and Tegegne Tesfa. 2022. Lexical complexity detection and simplification in amharic text using machine learning approach. In 2022 International Conference on Information and Communication Technology for Development for Africa (ICT4DA), pages 1–6. find
  9. Michael M. Woldeyohannis and Million Meshesha. 2022. Usable amharic text corpus for natural language processing applications. Applied Corpus Linguistics, 2(3):100033. ↗
  10. M. Zakaria. 2019. Text complexity classification based on linguistic information: Application to intelligent tutoring of esl. Journal of Data Mining and Digital Humanities, pages 1–40. find
  11. M. Read. 2019. Reading for Ethiopia S Achievement Developed Monitoring Usaid Reading for Ethiopia S Achievement Developed Monitoring. Usaid. find
  12. Mercado-Gonzales Rodolfo, Pereira-Noriega Jos, S. Cabezudo Marco, and Oncevay Arturo. 2018. Chanot: An intelligent annotation tool for indigenous and highly agglutinative languages in peru. In International Conference on Language Resources and Evaluation. find
  13. Santucci, V. Santarelli, F. Forti, and L. Spina. 2020. Automatic classification of text complexity. Applied Sciences, 10(20):1–19. ↗
  14. Namitha Seelam, Sanjan Pallerla, Reddy Nerella Chaithra, Srikar Yechuri, Shanmugasundaram Hariharan, and Andraju Bhanu Prasad. 2023. Sentiment analysis: Current state and future research perspectives. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), pages 1115–1119. find
  15. Y. Shanshan, S. Jindian, and L. Da. 2019. Improving bert-based text classification with auxiliary sentence and domain knowledge. IEEE Access, 7:176600– 176612. find
  16. Matthew Shardlow, Michael Cooper, and Marcos Zampieri. 2020. Complex: A new corpus for lexical complexity prediction from likert scale data. In Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI), pages 57–62. ↗
  17. Bott Stefan, Rello Luz, Drndarevic Biljana, and Saggion Horacio. 2012. Can spanish be simpler? lexsis: Lexical simplification for spanish. In International Conference on Computational Linguistics. find
  18. Lusito Stefano, Ferrante Edoardo, and Maillard Jean. 2022. Text normalization for low-resource languages: the case of ligurian. In COMPUTEL, pages 1–10. ↗
  19. Elior Sulem, Omri Abend, and Ari Rappoport. 2018. Semantic structural evaluation for text simplification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 685–696. ↗
  20. T. Vahe, D. John, W. Leigh, D. Alexander, R. Ziqin, K. Olga, P. Kristin A., C. Gerbrand, and J. Anubhav. 2019. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature, 571(7763):9598. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nigusie-2025-supervised-machine-learning-based-amharic/