Elamur Research⌕
☰
home / papers / 2016 / The Power of Language Music: Arabic Lemmatization through Patterns

The Power of Language Music: Arabic Lemmatization through Patterns

Paper · 2016 · Mohammed Attia, Ayah Zirikly and Mona Diab

Summary

The paper tackles Arabic lemmatization, meaning recovering the dictionary citation form of a word, without any lookup lexicon or morphological analyzer. The authors start from the traditional view that Arabic words are built by slotting root consonants into vocalic templates (patterns). They split the task in two. First, a classifier takes a tokenized, POS-tagged stem and predicts which lemma pattern it belongs to. Second, deterministic rules fill that pattern's slots with the stem's consonants to rebuild the vocalized lemma. They test two kinds of class inventory: a hand-built list of morpho-phonological patterns, and patterns derived automatically by masking the consonants of known lemmas. Experiments use diacritized and undiacritized versions of the Arabic Treebank (ATB). Against a baseline that returns the stem unchanged, results with diacritics are far higher; for verbs, accuracy rises from about 72% to about 98%. Without diacritics the picture is mixed. The work shows that a small set of abstract templates can replace a large dictionary, and the authors release their pattern list publicly.

Key points

  • The data comes from the ATB: 128,293 nouns, 31,666 verbs and 35,176 adjectives, split 80/10/10 into train, dev and test.
  • The released inventory holds 377 distinct patterns, which shrink to 175 once vowel marks are removed. Verb lemmas draw on 10 morphological and 35 phonological patterns, and broken plurals use 83 phonological patterns.
  • Decision Trees (C4.5) beat the Extra Trees classifier.
    • Diacritized verbs: the baseline of 72.35% rises to 98.23% for the final lemma, using C4.5 with the hand-made patterns.
    • Baselines for nouns and adjectives are 81.95% and 93.10%.
  • On undiacritized adjectives, pattern prediction stays high (mostly above 95%). However, the final pattern-to-lemma mapping falls below the stem baseline, which the authors attribute to weak mapping rules.
  • Statistics from BAMA show the lexicon is root-driven: about 87% of its 40,657 entries derive from 4,494 roots.

Interesting ideas

  • I1. The authors predict an abstract template class rather than the output string, then rebuild the lemma with simple deterministic slot-filling. This keeps the learned part small and leaves the generation step fully inspectable.
  • I2. Template classes can be generated automatically by masking the variable letters of known lemmas. These machine-made patterns performed about as well as the hand-curated list.
  • I3. The pattern slots are filled from the end of the stem toward the start, and weak letters that do not count as radicals are skipped. Because radicals keep their linear order, this alignment trick works well.
  • I4. Results are reported separately for the template-prediction step and the final reconstruction. This exposes where the errors come from, as in the undiacritized adjectives, where the classifier is fine but the rules fail.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): English lemmatization could predict an inflection-class label (e.g. "drop -ies → -y", "irregular: take→took") and apply a transparent rule, so every lemma comes with an explicit reason.
  • I2 (learn): Mova could induce its repair rules or morphology classes from silver lemma pairs by abstracting over the shared material, instead of hand-writing the class inventory.
  • I3 (tokens): aligning from the right edge (the suffix side) is a cheap, explainable way to map English inflected forms to stems when spelling changes occur near the end.
  • I4 (learn): evaluating each pipeline stage on its own, against a do-nothing baseline as a negative control, would show whether a regression comes from the learned model or the hand rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Dublin City UniversityDublin · Ireland70 works · 9,447 citations🎓George Washington UniversityWashington · United States51 works · 8,148 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Lemmatization · 20 works

Cite

Cite as Mohammed Attia, Ayah Zirikly and Mona Diab. 2016. The Power of Language Music: Arabic Lemmatization through Patterns. Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{attia2016power,
  title = {{The Power of Language Music: Arabic Lemmatization through Patterns}},
  author = {Mohammed Attia, Ayah Zirikly and Mona Diab},
  booktitle = {Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V)},
  year = {2016},
  url = {https://aclanthology.org/W16-5306/}
}

References

34 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Mona T Diab, and Sandra Kübler. 2013. Asma: A system for automatic segmentation and morpho-syntactic disambiguation of modern standard Arabic. In RANLP, pages 1–8.
  2. Mohamed Seghir Hadj Ameur, Youcef Moulahoum, and Ahmed Guessoum. 2015. Restoration of Arabic diacritics using a multilevel statistical model. In Computer Science and Its Applications, pages 181–192. Springer. find
  3. Mohammed Attia and Josef van Genabith. 2013. A jellyfish dictionary for Arabic. In Electronic lexicography in the 21st century: thinking outside the paper: proceedings of the eLex 2013 conference, 17-19 October 2013, Tallinn, Estonia, pages 195–212. find
  4. Vimala Balakrishnan and Ethel Lloyd-Yemoh. 2014. Stemming and lemmatization: a comparison of retrieval performances. Lecture Notes on Software Engineering, 2(3):262. ↗
  5. Mohamed Bebah, Chennoufi Amine, Mazroui Azzeddine, and Lakhouaja Abdelhak. 2014. Hybrid approaches for automatic vowelization of Arabic texts. arXiv preprint arXiv:1410.2646. ↗
  6. Karien Brits, Rigardt Pretorius, and Gerhard B van Huyssteen. 2005. Automatic lemmatization in Setswana: Towards a prototype. South African Journal of African Languages, 25(1):37–47. ↗
  7. Tim Buckwalter. 2004. Buckwalter Arabic morphological analyzer version 2.0. linguistic data consortium, university of pennsylvania, 2002. ldc cat alog no.: Ldc2004l02. Technical report, ISBN 1-58563-324-0. find
  8. João Ricardo Martins Ferreira da Silva. 2007. Shallow processing of Portuguese: From sentence chunking to nominal lemmatization. Master’s thesis. find
  9. Mona Diab. 2007. Improved Arabic base phrase chunking with a new enriched POS tag set. In In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, pages 89–96, Prague, Czech Republic.
  10. Mona Diab. 2009. Second generation AMIRA tools for Arabic processing: Fast and robust tokenization, pos tagging, and base phrase chunking. In In Proceedings of the Second International Conference on Arabic Language Resources and Tools, pages 285–288, Cairo, Egypt. find
  11. Tarek El-Shishtawy and Abdulwahab Al-Sammak. 2012. Arabic keyphrase extraction using linguistic knowledge and machine learning techniques. arXiv preprint arXiv:1203.4605. ↗
  12. Tarek El-Shishtawy and Fatma El-Ghannam. 2012. An accurate Arabic root-based lemmatizer for information retrieval purposes. arXiv preprint arXiv:1203.3584. ↗
  13. Tarek El-Shishtawy and Fatma El-Ghannam. 2014. A lemma based evaluator for semitic language text summarization systems. arXiv preprint arXiv:1403.5596. ↗
  14. Moustafa Elshafei, Husni Al-Muhtaseb, and Mansour Alghamdi. 2006. Statistical methods for automatic diacritization of Arabic text. In The Saudi 18th National Computer Conference. Riyadh, volume 18, pages 301–306. find
  15. Jan Hajic, Otakar Smrz, Tim Buckwalter, and Hubert Jin. 2005. Feature-based tagger of approximations of functional Arabic morphology. In Proceedings of the Workshop on Treebanks and Linguistic Theories (TLT), Barcelona, Spain. find
  16. Jan Hajič. 2000. Morphological tagging: Data vs. dictionaries. In Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference, pages 94–101. Association for Computational Linguistics.
  17. Faten Khalfallah Hammouda and Abdelsalam Abdelhamid Almarimi. 2010. Heuristic lemmatization for Arabic texts indexation and classification 1. find
  18. Ahmad Hossny, Khaled Shaalan, and Aly Fahmy. 2008. Automatic morphological rule induction for Arabic. In Proceedings of the LREC’08 workshop on HLT & NLP within the Arabic world: Arabic Language and local languages processing: Status Updates and Prospects, pages 97–101. find
  19. Matjaž Juršič, Igor Mozetič, and Nada Lavrač. 2007. Learning ripple down rules for efficient lemmatization. In Proceedings of the 10th international multiconference information society, IS, pages 206–209. find
  20. Dror Kamir, Naama Soreq, and Yoni Neeman. 2002. A comprehensive NLP system for modern standard Arabic and modern Hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages, pages 1–9. Association for Computational Linguistics.
  21. Katharina Kann and Hinrich Schütze. 2016. Single-model encoder-decoder with explicit morphological representation for reinflection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, pages 555–560. find
  22. Shereen Khoja and Roger Garside. 1999. Stemming Arabic text. Lancaster, UK, Computing Department, Lancaster University. find
  23. Tuomo Korenius, Jorma Laurikkala, Kalervo Järvelin, and Martti Juhola. 2004. Stemming and lemmatization in the clustering of Finnish text documents. In Proceedings of the thirteenth ACM international conference on Information and knowledge management, pages 625–633. ACM. find
  24. Leah S Larkey, Lisa Ballesteros, and Margaret E Connell. 2002. Improving stemming for Arabic information retrieval: Light stemming and co-occurrence analysis. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pages 275–282. ACM. find
  25. Rani Nelken and Stuart M Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 79–86. Association for Computational Linguistics.
  26. Okan Ozturkmenoglu and Adil Alpkocak. 2012. Comparison of different lemmatization approaches for information retrieval on Turkish text collection. In Innovations in Intelligent Systems and Applications (INISTA), 2012 International Symposium on, pages 1–5. IEEE. find
  27. Joël Plisson, Nada Lavrac, Dunja Mladenic, et al. 2004. A rule based approach to word lemmatization. Proceedings of IS-2004, pages 83–86. find
  28. Mohsen Rashwan, Mohamed Al-Badrashiny, Mohamed Attia, Sherif Abdou, and Ahmed Rafea. 2011. A stochastic Arabic diacritizer based on a hybrid of factorized and unfactorized textual features. Audio, Speech, and Language Processing, IEEE Transactions on, 19(1):166–175. ↗
  29. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pages 117–120. Association for Computational Linguistics.
  30. Djamé Seddah, Grzegorz Chrupała, Özlem Çetinoğlu, Josef Van Genabith, and Marie Candito. 2010. Lemmatization and lexicalized statistical parsing of morphologically rich languages: The case of French. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 85–93. Association for Computational Linguistics. find
  31. Nasredine Semmar, Meriama Laib, and Christian Fluhr. 2006. Using stemming in morphological analysis to improve Arabic information retrieval. In TALN 2006, pages 317–326, Leuven, Belgium.
  32. Lucie Skorkovská. 2012. Application of lemmatization and summarization methods in topic identification module for large scale language modeling data filtering. In Text, Speech and Dialogue, pages 191–198. Springer. find
  33. Marko Tadić. 2006. Croatian lemmatization server. In Fifth International Conference Formal Approaches to South Slavic and Balkan languages (FASSBL). find
  34. Kristina Toutanova and Christopher D Manning. 2000. Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics-Volume 13, pages 63–70. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attia-2016-power-language-music-arabic-lemmatization/