Elamur Research⌕
☰
home / papers / 1996 / Arabic Finite-State Morphological Analysis and Generation

Arabic Finite-State Morphological Analysis and Generation

Paper · 1996 · Kenneth R. Beesley

Summary

Beesley tackles automatic morphological analysis and generation for Arabic. Arabic stems are built by interleaving a consonantal root with a vowel-and-slot pattern, and ordinary text usually leaves out short vowels. As a result, a written word has on average almost five valid analyses. The starting point was an earlier KIMMO-style two-level analyzer built at ALPNET. That system was slow and awkward to use for generation. In 1995 its lexicons and rules were moved into the Xerox finite-state toolkit (lexc, twolc). The key observation is that combining a root with a pattern is simply finite-state intersection, so it can be done at compile time rather than at runtime. Roots, patterns, affixes and alternation rules are then intersected and composed into one Arabic Lexical Transducer. Analyses expose the root plus feature tags. The system accepts unvoweled, partially voweled and fully voweled input, and it generates surface forms by running the same machine in reverse. It also uses the same runtime engine as the Xerox transducers for major European languages, showing that "exotic" morphology fits ordinary finite-state machinery.

Key points

  • Scale of the lexicon:
    • about 5000 roots and roughly 400 phonologically distinct patterns;
    • each root is coded to allow only its legal patterns, on average about 17–18;
    • about 85,000 valid stems are produced by intersection at compile time.
  • The old ALPNET system (1989–90) had several weaknesses:
    • it built stems at runtime with an algorithm called Detouring;
    • its more than 100 hand-compiled rule transducers were run in parallel;
    • running on a small IBM mainframe, it could process only around 2 words per second.
  • Reimplementation pipeline:
    • lexicons were converted to lexc;
    • rules were recompiled and intersected into one rule FST with twolc;
    • the rule FST was composed under the lexicon FST;
    • a filter transducer composed on top removes rule-trigger features and enforces long-distance morphotactic constraints.
  • Output format and input coverage:
    • the upper side of each analysis is a root followed by tags such as +FormI, +Perfect, +Active/+Passive, person and number;
    • input may use real Arabic encodings (ASMO 449, ISO8859-6) rather than a lossy romanization.
  • Generation of only fully voweled forms uses a separate rule set. Composing both rule sets restores the general recognizer.

Interesting ideas

  • I1. Non-concatenative word formation (root plus pattern) can be modeled as intersecting two regular languages, which turns an unusual process into a standard compile-time operation.
  • I2. Compiling the lexicon and every alternation rule into one transducer removes the cost of consulting many machines step by step, and it also eliminates building and then discarding phantom candidates.
  • I3. Make the upper side of each analysis a baseform plus explicit feature symbols, so that analysis and generation are exact inverses of each other.
  • I4. Keep strict and lenient surface rule sets separately: one that accepts only fully specified forms, and one that also tolerates underspecified input. Compose them for recognition, and use the strict one for output.
  • I5. Each lexical entry is coded with exactly which patterns it licenses, which controls overgeneration through explicit lexicography rather than statistics.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Irregular or templatic English phenomena, such as strong-verb ablaut, could be described as constraints that are intersected or composed offline. This would keep runtime lookup a single deterministic pass that can be explained.
  • I2 (tokens): The en crate's lemmatizer and rule layers could be precompiled into one FST, so that speed no longer depends on the number of rules.
  • I3 (back): Generation from the meaning graph could produce lemma+tag strings and run the same morphology transducer backwards. Surface forms would then be guaranteed to match what the analyzer accepts.
  • I4 (back): Mova could analyze noisy or informal spellings leniently while generating only canonical forms.
  • I5 (world): Verb-class and frame entries compiled from VerbNet could likewise list allowed alternations explicitly, keeping ungrammatical combinations out of the knowledge base.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 works

Cite

Cite as Kenneth R. Beesley. 1996. Arabic Finite-State Morphological Analysis and Generation. COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{beesley1996arabic,
  title = {{Arabic Finite-State Morphological Analysis and Generation}},
  author = {Kenneth R. Beesley},
  booktitle = {COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics},
  year = {1996},
  doi = {10.3115/992628.992647},
  url = {https://doi.org/10.3115/992628.992647}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antworth, Evan L. 1990. PC-KIMMO: A Two-level Processor for Morphological Analysis. Occasional Publications in Academic Computing No. 16. Dallas: Summer Institute of Linguistics. find
  2. Beesley, Kenneth R. 1989. Computer Analysis of Arabic Morphology: A Two-Level Approach with Detours. Read at the Third Annual Symposium on Arabic I,inguistics, University of Utah, Salt Lake City, Utah, 3-4 March 1989. Published in Bernard Comrie and Mushira Eid (eds.), Perspectives on Arabic Linguistics 111: l'apers from the Third Anuual Symposium on Arabic Linguistics, Amsterdam: John Benjamins, pp. 155-172. find
  3. Beesley, Kcnneth R.; Buckwalter, Tim; and Newton, Stuart N. 1989. Two-Level Finite-State Analysis of Arabic Morphology. In Proceedings of the Seminar on Bilingual Computing in Arabic and English, Cambridge, England, 6-7 Sept 1989. find
  4. No pagination. Beesley, Kenneth R.. 1990. Finite-State Description of Arabic Morphology. In Proceedings of the Second Cambridge Conference on Bilingual Computing in Arabic and English, 5-7 Sept 1990. find
  5. No pagination. Beeston, A.F.L. 1968. Written Arabic: an approach to the basic structures. Cambridge: Cambridge University Press. find
  6. Buckwalter, Timothy A. 1990. Lexicographic Notation of Arabic Noun Pattern Morphemes and Their Inflectional Features. In Proceedings of the Second Cambridge Conference on Bilingual Computing in Arabic and English, 5-7 Sept 1990. No pagination. Gajek, Oliver el al. 1983. LISP lmplcmenta-tion. Texas Linguistic Forum 22 ed. by 1)alrym-pie et al. Austin: Linguistics Department. University of Texas, pp. 187-202 Kaplan, Ronald M. and Kay, Martin. 1981. l)honological rules and linite-state transducers [Abstract]. Linguistic Society of America Meeting Handbook. Fifty-Sixth Annual Meeting, December 27-30, 1981. New York. find
  7. Kaplan, Ronald M. and Kay, Martin. 1994. find
  8. Regular Models of Phonological Rule Systems. Computational Linguistics. 20:3, pp. 331-378. Karttunen, Lauri. 1983. A General Morphological Processor. Texas Linguistic Forum 22 ed. by Dalrumple et al. Austin: Linguistics Department, University of Texas, pp. 165-186. find
  9. Karttunen, Lauri. 1991. Finite-State Constraints. In the Proceedings of the International Conference on Current Issues in Computational Linguistics. June 10-14, 1991. Penang:Universiti Sains Malaysia. find
  10. Karttunen, Lauri; Kaplan, Ronald M.; and Zaenen, Annie. 1992. Two-Level Morphology with Composition. COLING 92, pp. 141-148. find
  11. Karttunen, Lauri. 1993. Finite-State Lexicon Compiler. Technical Report. ISTL-NLTT-1993-04-02. Xerox Palo Alto Research Center. Palo Alto, California. find
  12. Koskenniemi, Kimmo. 1983. Two-Level Morphology: A General Computational Model for Word-Form Recognition and Production. Publication No. 11. Helsinki: Department of General Linguistics, University of Helsinki. find
  13. Koskenniemi, Kimmo. 1984. A General Computational Model for Word-Form Recognition and Production. COLING 84, pp. 178-181. find
  14. Karttunen, Lauri and Beesley, Kenneth R. 1992. Two-Level Rule Compiler. Technical Report. ISTL-1992-2. Xerox Palo Alto Research Center. Palo Alto, California. find
  15. McCarthy, J. 1981. A Prosodic Theory of Non-concatenative Morphology. Linguistic Inquiry, 12(3), pp. 373-418. find
  16. Wehr, Hans. 1976. A Dictionary of Modern Written Arabic. Third edition, ed. by J. Milton Cowan. Ithaca, N.Y.:Spoken Language Services, Inc. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/beesley-1996-arabic-finite-state-morphological-analysis/