Elamur Research⌕
☰
home / papers / 2006 / Standard Arabic Formalization and Linguistic Platform for its Analysis

Standard Arabic Formalization and Linguistic Platform for its Analysis

Paper · 2006 · Slim Mesfar

Summary

The paper tackles automatic morphological analysis of standard Arabic. Two properties make this hard: words agglutinate prefixes, a stem and suffixes, and short vowels are usually left out of writing, which produces heavy ambiguity. The author builds an Arabic module for NooJ, a language-independent finite-state platform. The module is designed to handle fully vowelled, partially vowelled and unvowelled text. It does not generate words from roots and patterns, because that approach can invent lemmas that do not exist or miss real ones. Instead it relies on hand-built lemma dictionaries. Each lemma is linked to an inflectional or derivational paradigm, written as a sequence of cursor-editing commands that turn the lemma into its surface forms in linear time. Three custom operators let near-identical paradigms share one description. Nouns come from a dictionary of primitive nouns plus deverbal forms derived automatically from the verbs. The result is a large-coverage lexical analyzer that can be inspected. Its weak spots are named entities and badly vowelled input, and the author is building a recognizer and fallback grammars for these. The work matters as a reusable, rule-based foundation for further Arabic NLP.

Key points

  • The verb dictionary has 10,000 fully vowelled entries. Each entry is a 3rd person masculine singular perfect form, linked to one of 130 hand-encoded inflectional paradigms, and yields about 122 inflected forms on average.
  • Paradigms encode mood, voice, gender, number and person using NooJ string commands such as <LW>, <R>, <S> and <B>. These run in O(n) time.
  • The author added three operators:
    • <T> rewrites Teh marbouta to Teh maftouha, for example in duals.
    • <Z> and <M> allow verbs with slightly different past-tense behaviour, such as kataba vs. Ťabata and kataba vs. dahana, to share a single paradigm.
  • The noun dictionary has about 15,000 primitive nouns stored without their final vowel. It also includes plurals that lack a singular in use. Deverbals (IsmFaîl, IsmMafoûl, Masdar) are derived from verb entries through DRV descriptions, for example mudarris and mudarras from darrasa.
  • Most unrecognized forms are named entities. In response, the author is building an NER module with NooJ's syntactic grammars and low-priority grammars that strip vowels when a form is unknown.

Interesting ideas

  • I1. Build the lexicon from attested lemmas with assigned paradigms instead of freely combining roots and patterns, so the system never produces "virtual" words.
  • I2. Write each inflection as a short program of cursor moves, deletions and insertions applied to the lemma, which makes every generated form traceable step by step.
  • I3. Add conditional operators that inspect the stem, such as its final letter, so near-identical inflection classes can be merged into one paradigm instead of multiplying classes.
  • I4. Apply relaxed grammars only when a form fails normal analysis, and relax step by step: first remove the last vowel, then all vowels.
  • I5. Derive deverbal nouns such as agent and patient nouns directly from verb entries, so each noun keeps a link to its source verb and the role it fills.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): English morphology could store real lemmas with paradigm IDs rather than productive suffix rules, which cuts overgenerated analyses like nonexistent derived forms.
  • I2 (tokens): spelling changes such as y→ies or consonant doubling could be small edit scripts attached to paradigms, giving each lemma↔form link a readable derivation.
  • I3 (tokens): a few stem-sensitive operators could merge English irregular-ish verb classes and keep the paradigm inventory small and auditable.
  • I4 (tokens): unknown English tokens could go through a priority-ordered repair cascade (case, diacritics, common misspellings), with each relaxation logged as the explanation.
  • I5 (frames): linking "teacher" to "teach" as agent (and similar nominalizations) would let prag fill VerbNet roles from noun phrases as well as from verbs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksMorphology · 420 works

Cite

Cite as Slim Mesfar. 2006. Standard Arabic Formalization and Linguistic Platform for its Analysis. Proceedings of the International Conference on the Challenge of Arabic for NLP/MT.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mesfar2006standard,
  title = {{Standard Arabic Formalization and Linguistic Platform for its Analysis}},
  author = {Slim Mesfar},
  booktitle = {Proceedings of the International Conference on the Challenge of Arabic for NLP/MT},
  year = {2006},
  url = {https://aclanthology.org/2006.bcs-1.8/}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abou Il Azm, A.(2003) Tasrif Moojim il afál: 10 000 verbs. Dar Ittawhidi, Rabat. find
  2. Abdeli, A., Cowie, J., Soliman, H. (2004) Arabic Information Retrieval Perspectives. JEP - TALN Session on the automatic treatment of the written and spoken Arabic language, Morocco. find
  3. Achour, H. (1998) Contribution à l’étude du problème de la voyellation automatique de l’arabe. PhD Thesis, Paris 7 University. find
  4. Beesley, K., Buckwalter, A. (1989) Two-level Finite State Analysis for Arabic Morphology. In Proceedings of the Seminar on Bilingual Computing in Arabic and English. The Literary and Linguistic Computing Centre, Cambridge. find
  5. Beesley, K. (1996) Arabic Finite-State Morphological Analysis and Generation. In Proceedings of COLING-96, the16th International Conference on Computational Linguistics, Copenhagen.
  6. Beesley, K. (1998) Arabic Morphology Using Only Finite-State Operations. In M. Rosner, editor, Proceedings of the Workshop on Computational Approaches to Semitic Languages, Montreal.
  7. Beesley, K. (2001) Arabic Finite-State Morphological Analysis and Generation of Arabic at Xerox Research: Status and Plans in 2001. In Proceedings of ACL/EACL 2001 Workshop, ARABIC Language Processing: Status and Prospects, Toulouse. find
  8. Chomsky, N. (1971) Aspects of the Theory of Syntax. Trad. J.-C. Milner. find
  9. Debili, F. (2001) Traitement automatique de l’arabe voyellé ou non. Correspondances – IRMC. find
  10. Debili, F., Achour, H., and Soussi, E. (2002) La langue arabe et l’ordinateur : de l’étiquetage grammatical à la voyellation automatique. Correspondances – IRMC. find
  11. Dichy, J., and Farghaly, A. (2003) Roots & Patterns vs. Stems plus Grammar-Lexis Specifications: there what basis should is multilingual lexical database centred on Arabic be built? Workshop on Machine Translation for Semitic Languages, New Orleans, USA. find
  12. Dichy, J. (2001) On lemmatization in Arabic. A formal definition of the Arabic entries of multilingual lexical databases. In proceedings of ACL/ EACL 2001. find
  13. Harris, Z. S. (1985) Transformational Theory. Language, vol. 41, No. 9, pp. 363-401. find
  14. Khoja, S., Garside, R., and Knowles, G. (2001) A tagset for the morpho-syntactic tagging of Arabic. Paper given At the Corpus Linguistics 2001 conference, Lancaster. find
  15. Koskenniemi, K. (1983) Two-level morphology: a general computational model for word-form recognition and publication. University of Helsinki. find
  16. McCarthy, J. (1993) Template form in prosodic morphology. Paper from the Third Annual Formal Linguistics Society of Midamerica Conference, pages 187-218, Bloomington, Indiana. Indiana University Linguistics Club. find
  17. Mesfar, S. (2006) La reconnaissance des entités nommées (NER) en arabe standard. The 9th INTEX/NooJ conference, The Faculty of Mathematics, University of Belgrade. find
  18. Revuz, D. (1991) Dictionnaires et lexiques : méthodes et algorithmes. PhD Thesis, Paris 7 University. find
  19. Silberztein, M. (2005_1) NooJ’s Dictionnaries. In proceedings of LTC 2005, Pozman University. Silberztein, M. (2005_2) NooJ : The Lexical Module In NooJ pour le Traitement Automatique des Langues. S. Koeva, D. Maurel, M. Silberztein Eds, MSH Ledoux. find
  20. Franche-Comté Academic Presses, 2005. find
  21. Silberztein, M. (2006) NooJ Manual. Download from “http://www.nooj4nlp.net” ↗
  22. Ouersighni, R. (2001) A major offshot of the DIINAR-MBC project: AraParse, a morpho-syntactic analyzer for unvowelled Arabic texts. In proceedings of ACL/EACL 2001. find
  23. Tuerlinckx, L. (2004) La lemmatisation de l’arabe non classique. The 7 th international Days of Statistical Analysis of Textual Data, JADT. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mesfar-2006-standard-arabic-formalization-linguistic-platform/