📄 Induction of Root and Pattern Lexicon for Unsupervised Morphological Analysis of Arabic

Summary

The paper addresses how to learn Arabic non-concatenative morphology without supervision. In Arabic, a word arises from a consonantal root interleaved with a pattern template, not from a linear chain of morphemes. Khaliq and Carroll take each stemmed word and enumerate every possible split into a trilateral root and its leftover pattern. The two parts then score each other in alternation: a pattern earns credit from the roots it co-occurs with, and a root earns credit from its patterns. The authors note this resembles Kleinberg's hubs and authorities algorithm. Scores are rescaled so the two lexicons are comparable. A refinement phase then averages, rather than sums, the partner scores. To analyse a word, the system picks the root–pattern split with the highest combined score.

Unlike earlier work, which used artificial text or phonetic transcriptions, this method runs on ordinary unvowelled script. On stemmed vocabulary from the Quranic Arabic Corpus, root identification rises from about 74% for a frequency baseline to 94.3%. This suggests template morphology can be learned from raw text alone.

Key points

• The evaluation data comes from the Quranic Arabic Corpus (QAC), which has about 7370 undiacritized stemmed types, roughly 95% of them annotated with roots. Words with hollow or reduplicated roots were excluded as out of scope, leaving 5468 word–root pairs.
• The plain frequency baseline identifies about 74% of roots correctly (4055 words).
• Five rounds of mutual rescoring raise accuracy to 83.0%, and the first round alone adds about 7 points.
• Five rounds of average-based refinement on top bring accuracy to 93.6%, again with a jump of about 7 points on the first round.
• The IF1 setting keeps only patterns with single-letter infixes, which fits text without short vowels. It reaches the best result, 94.3% (5159 correct), more than 20 points above the baseline.

Interesting ideas

• I1. Two kinds of units that explain each other can be ranked together, each scoring the other in a loop, in the style of hubs and authorities; this works without labels whenever pairs of units co-occur.
• I2. Before adding scores that come from different distributions, rescale one by the ratio of the two maxima so neither score swamps the other.
• I3. Switching from summed partner scores to averaged partner scores separates genuine units from merely frequent ones, because a unit is judged by the quality of its company rather than its volume.
• I4. Enumerating all possible decompositions and then pruning them with a cheap linguistic constraint (here, single-letter infixes in unvowelled text) removes junk analyses before scoring.
• I5. The evaluation explicitly sets aside cases the model cannot represent (hollow and reduplicated roots) and reports how many remain. This keeps the accuracy figure honest about the method's scope.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Verb classes and the entity types that fill their slots could reinforce each other this way, yielding ranked selectional preferences from parsed text without annotation.
• I2 (learn): When merging confidences from several witness parsers or LLM teacher tiers, this simple max-ratio scaling would keep one noisy source from dominating the silver-data vote.
• I3 (learn): Self-induced repair rules could be ranked by the average correctness of the sentences they fire on rather than by firing count, which would demote frequent but harmful rules.
• I4 (tokens): The morphology and lemma module could generate every suffix-stripping hypothesis and filter it with English orthographic rules before ranking, with each rejected option kept as an explainable trace.
• I5 (answer): Test reports could likewise separate questions that need capabilities Mova lacks from in-scope ones, so scores reflect real competence and gaps stay visible.