📄 A Unification based Approach to the Morphological Analysis and Generation of Arabic

Summary

Köprü and Miller describe a combined morphological analyzer and generator for Modern Standard Arabic. Arabic morphology is non-concatenative, because affix material can be interleaved with root letters. It is also hard because some morphemes, such as nominal number, have very many surface realizations. The system is a finite-state transducer whose arcs carry unification-based feature structures. Grammar writers do not build the automaton directly. They write high-level rules over morpheme categories, and each category pairs a feature structure with an allomorph table of string operations. These rules are then compiled into the FST. A stem-based lexicon of 75K entries supplies rich grammatical features. Generation reuses the analysis grammar by running the compiled machine in reverse. The authors tested the analyzer by hand on 500 tokens of the Penn Arabic Treebank (ATB) and report 91.4% accuracy. Most errors come from words missing from the lexicon. The analyzer also proposes far fewer candidate analyses than the Buckwalter analyzer. This matters because the same component serves both statistical and rule-based MT.

Key points

• The core is an FST with feature structures stored as DAGs, and unification decides whether a morpheme matches. A morpheme counts only if one of its allomorph string operations succeeds and its features unify with those of the stem.
• The lexicon holds 75K stem-keyed entries that also record inflection patterns. Predictable forms, such as feminine nouns derived from masculine ones, are not stored separately.
• Gerunds are mapped back to their verbal stems through about 30 derivation patterns, chosen via a DPATTERN label in the verb's entry. The verb's PRED information is kept in the result.
• Evaluation used the first 500 tokens of ATB section 20,000,915 and was scored manually against the gold annotation. Overall accuracy was 91.4%.
  • Out-of-lexicon words account for 72% of errors, and wrong analyses for the remaining 28%.
  • The system produced 689 candidate analyses where Buckwalter produced 5,286.
• The same framework has also been used for English, Korean and Turkish.

Interesting ideas

• I1. A rule's order lists the sequence in which morpheme operations are applied, not where the pieces sit in the string, so infixes and circumfixes fit the same formalism as suffixes.
• I2. Each string operation has an exact inverse (cut ↔ add, geminate ↔ de-geminate), so one grammar compiled once can both analyze and generate by reversing the machine.
• I3. Unification acts as a filter: an affix whose features clash with the stem's lexical features is simply rejected. This prunes candidates heavily compared with an analyzer that is not driven by the lexicon.
• I4. Derived nouns such as gerunds are traced back to their source verb, and the verb's argument frame is carried along rather than copied into a separate lexicon entry.
• I5. The error analysis separates missing-lexicon failures from wrong analyses and also counts how many candidates were produced.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): English irregulars (sang/sing, geese/goose) could be described as ordered stem operations inside one rule format, instead of being handled as a separate exception list.
• I2 (back): Mova's surface realizer could invert the same morphology rules the tagger uses, which guarantees that what is generated agrees with what the analyzer accepts.
• I3 (tokens): checking lemma features against suffix features (e.g., a mass noun with plural -s) would shrink the candidate set the tagger has to choose from, and each rejection would have an explainable reason.
• I4 (frames): English nominalizations ("the destruction of the city") could inherit the VerbNet roles of their base verb, so "who did what to whom" still holds inside noun phrases.
• I5 (learn): morphology evaluations could report coverage gaps, wrong analyses and ambiguity load as three separate numbers, so each fix can be aimed at the right part of the system.