Elamur Research⌕
☰
home / papers / 2006 / An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modeling Finite State Networks

An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modeling Finite State Networks

Paper · 2006 · Mohammed A. Attia

Summary

Attia targets morphological ambiguity in Modern Standard Arabic. Every extra analysis a morphological analyzer returns slows downstream parsers and taggers and multiplies their outputs. He contends that design flaws inflate ambiguity in the two most widely known analyzers: the Buckwalter Arabic Morphological Analyzer and the finite-state morphology system developed by Xerox for Arabic. Xerox is root-based and overgenerates. Buckwalter is a hand-listed stem database. Both include classical vocabulary absent from modern text.

His alternative is a rule-based finite-state analyzer that uses the stem as its base form. Its lexicon is drawn from a contemporary news corpus. It enforces constraints on how words combine with clitics, applies spelling relaxation conservatively, and licenses passive and imperative forms only for verbs judged to accept them. The paper also lists the legitimate sources of Arabic homography and proposes an "ambiguity pyramid": ambiguity falls as affixes and clitics are added.

In a manual test on Al-Jazeera text, the new analyzer produced far fewer analyses than either competitor and had higher precision than Buckwalter. Attia concludes that rich morphology need not imply high ambiguity.

Key points

  • Test data: five recent Al-Jazeera documents containing 950 unique words and 67 MWEs. After removing the 55 words unknown to any system, 895 words were compared.
  • Total analyses returned for the 895 words:
    • Xerox: 3871
    • Buckwalter: 2332
    • Attia: 1574, roughly 1.76 per word by this summary's own calculation.
    • An English baseline from the XLE transducer on 979 words also averaged 1.76 analyses per word.
  • Precision: "complete" analyses (no spurious readings and no missing ones) reached 79% for Attia versus 64% for Buckwalter. This holds even though Attia's lexicon is about a quarter the size of Buckwalter's 38,600 lemmas.
  • Verb licensing: only 483 of 1297 verbs (37%) may take the imperative. 198 transitive verbs are denied a passive. First- and second-person passives are blocked entirely.
  • MWEs and error sources: Attia's system recognized 25 of the 67 MWEs (37%). Error review traced the competitors' spurious readings to three causes:
    • classical entries,
    • ignored word–clitic combination rules,
    • overused spelling relaxation.

Interesting ideas

  • I1. Separate legal ambiguity (genuine homographs) from spurious ambiguity (readings the language does not actually license), and grade each output as complete, over-specified, under-specified, or both; this exposes whether a system errs by excess or by omission.
  • I2. Decide per verb, guided by semantic classes from Levin (perception verbs, entity-specific change of state), whether a construction such as the imperative or passive is allowed, instead of generating every inflection for every verb.
  • I3. Restrict the lexicon to vocabulary attested in a contemporary target-domain corpus; archaic entries silently add readings that are never correct in practice.
  • I4. The ambiguity pyramid: analyze the same text at stem, inflected, and fully cliticized levels, and watch ambiguity shrink as more material is attached.
  • I5. Calibrate a system's ambiguity rate against a reference point, here an English analyzer's rate, rather than reporting the number in isolation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Use these four categories when scoring tagger and lemmatizer output against silver data, so that noise and gaps in the candidate set are tracked as separate failure modes.
  • I2 (tokens): Let VerbNet/Levin class membership suppress unlikely morphological or construction readings in the en crate, cutting candidate analyses before they reach the parser.
  • I3 (world): When compiling WordNet-derived senses, down-weight or prune senses unattested in the target genre (school texts, fables) to shrink sense ambiguity in the world model.
  • I4 (tree): Delay committing to a part of speech until affix and attachment context is available, treating bare forms as maximally ambiguous.
  • I5 (learn): Report candidate counts per token next to a reference analyzer's counts on the same text, as a sanity baseline for overgeneration.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 works

Cite

Cite as Mohammed A. Attia. 2006. An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modeling Finite State Networks. Proceedings of the International Conference on the Challenge of Arabic for NLP/MT.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{attia2006ambiguity,
  title = {{An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modeling Finite State Networks}},
  author = {Mohammed A. Attia},
  booktitle = {Proceedings of the International Conference on the Challenge of Arabic for NLP/MT},
  year = {2006},
  url = {https://aclanthology.org/2006.bcs-1.5/}
}

References

26 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbès R, Dichy J, Hassoun M (2004): The Architecture of a Standard Arabic lexical database: some figures, ratios and categories from the DIINAR.1 source program, The Workshop on Computational Approaches to Arabic Script-based Languages, COLING 2004. Geneva, Switzerland.
  2. Attia MA (2005): Developing a Robust Arabic Morphological Transducer Using Finite State Technology, 8th Annual CLUK Research Colloquium. Manchester, UK. find
  3. Attia MA (2006): Accommodating Multiword Expressions in an Arabic LFG Grammar. In Salakoski T, Ginter F, Pyysalo S (eds), FinTAL 2006, Vol 4139. Turku, Finland: Springer-Verlag Berlin Heidelberg, pp 87 - 98. find
  4. Azmi M (1988): Arabic Morphology: A Study in the System of Conjugation. Hyderabad: Hasan Publishers. find
  5. Beesley KR (1996): Arabic Finite-State Morphological Analysis and Generation, Proceedings of the 16th conference on Computational linguistics, Vol 1. Copenhagen, Denmark: Association for Computational Linguistics, pp 89-94.
  6. Beesley KR (1998a): Arabic Morphological Analysis on the Internet, The 6th International Conference on Multilingual Computing. Cambridge. find
  7. Beesley KR (1998b): Arabic Morphology Using Only Finite-State Operations, Proceedings of the Workshop on Computational Approaches to Semitic languages. Montreal, Quebec, pp 50-57.
  8. Beesley KR (2001): Finite-State Morphological Analysis and Generation of Arabic at Xerox Research: Status and Plans in 2001, Proceedings of the Arabic Language Processing: Status and Prospect — 39th Annual Meeting of the Association for Computational Linguistics. Toulouse, France. find
  9. Beesley KR, Karttunen L (2003): Finite State Morphology. Stanford, Calif.: Csli. find
  10. Buckwalter T (2004): Issues in Arabic Orthography and Morphology Analysis, The Workshop on Computational Approaches to Arabic Script-based Languages, COLING 2004. Geneva.
  11. Butt M, Dyvik H, King TH, Masuichi H, Rohrer C (2002): The Parallel Grammar Project, COLING-2002 Workshop on Grammar Engineering and Evaluation. Taipei, Taiwan.
  12. Chalabi A (2004): Sakhr Arabic Lexicon, NEMLAR International Conference on Arabic Language Resources and Tools. Cairo, Egypt. find
  13. Darwish K (2002): Building a Shallow Morphological Analyzer in One Day, Proceedings of the workshop on Computational Approaches to Semitic Languages in the 40th Annual Meeting of the Association for Computational Linguistics (ACL-02). Philadelphia, PA, USA. find
  14. Dichy J (2001): On lemmatization in Arabic. A formal definition of the Arabic entries of multilingual lexical databases, ACL 39th Annual Meeting. Workshop on Arabic Language Processing; Status and Prospect. Toulouse, pp 23-30. find
  15. Dichy J, Fargaly A (2003): Roots & Patterns vs. Stems plus Grammar-Lexis Specifications: on what basis should a multilingual lexical database centred on Arabic be built?, Proceedings of the MT-Summit IX workshop on Machine Translation for Semitic Languages. New-Orleans. find
  16. Dichy J, Hassoun M (1998): Some aspects of the DIINAR-MBC research programme, Proceedings of 6th ICEMCO, International Conference and Exhibition on Multilingual Computing. Cambridge, England. find
  17. Freeman A (2001): Brill's POS tagger and a Morphology parser for Arabic, 39th Annual Meeting of Association for Computational Linguistics & 10th Conference of the European Chapter, Workshop on Arabic Language Processing: Status and Prospects. Toulouse, France. find
  18. Hajic J, Smrž O, Buckwalter T, Jin H (2005): Feature-Based Tagger of Approximations of Functional Arabic Morphology, The Fourth Workshop on Treebanks and Linguistic Theories. Universitat de Barcelona. find
  19. Ibrahim K (2002): Al-Murshid fi Qawa'id Al-Nahw wa Al-Sarf [The Guide in Syntax and Morphology Rules]. Amman, Jordan: Al-Ahliyyah for Publishing and Distribution. find
  20. Kamir D, Soreq N, Neeman Y (2002): A Comprehensive NLP System for Modern Standard Arabic and Modern Hebrew, Proceedings of the workshop on Computational Approaches to Semitic Languages in the 40th Annual Meeting of the Association for Computational Linguistics (ACL-02). Philadelphia, PA, USA.
  21. Kiraz GA (1998): Arabic Computational Morphology in the West, Proceedings of the 6th International Conference and Exhibition on Multi-lingual Computing. Cambridge, UK. find
  22. Levin B (1993): English Verb Classes and Alternations: A Preliminary Investigation: University of Chicago Press. find
  23. McCarthy JJ (1985): Formal Problems in Semitic Phonology and Morphology. New York ; London: Garland. find
  24. Ratcliffe RR (1998): The Broken Plural Problem in Arabic and Comparative Semitic : Allomorphy and Analogy in Non-concatenative Morphology. Amsterdam ; Philadelphia: J. Benjamins. find
  25. Wehr H (1979): A Dictionary of Modern Written Arabic, 4th Edition ed. Ithaca, NY: Spoken Language Services, Inc. find
  26. Yona S, Wintner S (2007): A finite-state morphological grammar of Hebrew. Natural Language Engineering. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attia-2006-ambiguity-controlled-morphological-analyzer-modern/