📄 Unsupervised Induction of Arabic Root and Pattern Lexicons using Machine Learning

Summary

The paper addresses unsupervised learning of non-concatenative morphology. In Arabic, a consonantal root is interleaved with a pattern template, so there is no clean boundary at which to split a word. The authors work on unvowelled, lightly stemmed words from the Quranic Arabic Corpus (QAC), which matches how Arabic is normally written. Each word is broken into every possible pairing of a three-letter root and its leftover pattern.

A Maximum Entropy classifier is trained with each word as its own class, using orthographic features of two kinds: character subsets for roots and placeholder templates for patterns. The model is then run back over the same vocabulary to get a proximity ranking between words. Neighbours that share patterns supply evidence for roots, and neighbours that share roots supply evidence for patterns. Scores gathered from these clusters build a root lexicon and a pattern lexicon, and each word gets the highest-scoring root–pattern pair.

With log scaling, length adjustment and a structural constraint, root extraction reaches 87.2% accuracy, against a 74.2% count-based baseline. This matters because it works on naturally written text without vowels and needs little linguistic knowledge.

Key points

• The data is the QAC: about 77,900 tokens and about 19,000 unique forms. Removing diacritics leaves about 14,850 forms, and light stemming leaves about 7,370. After excluding hollow and reduplicated roots, 5,468 stemmed types are evaluated.

• The approach follows De Pauw and Wagacha (2007) in using a supervised ME classifier without supervision. Training uses L-BFGS and stops once accuracy on the training set reaches 100%. Each headword's cluster is cut at its 500 nearest neighbours.

• Feature design:

  • Words are wrapped in boundary markers ‘@’ and ‘#’.
  • Root features are built from the power set of characters in the word.
  • Pattern features are the inverse view, with root radicals replaced by placeholders.

• Root accuracy by configuration:

  Configuration	Accuracy
  Baseline	74.16%
  ME_RW (raw probabilities)	65.78%
  ME_LS (log-scaled)	80.74%
  ME_LS_LA (+ length adjustment)	85.95%
  ME_LS_LA_NC1 (+ single-affix constraint)	87.20%

• The top entries in the induced pattern lexicon are prefix templates such as y — , t — and m — , and templates with an infix A such as --A- and -A — .

Interesting ideas

• I1. Treat every vocabulary item as its own class, train a classifier on its features, then score the training items against each other; the output distribution acts as a learned similarity that captures shared morphemes better than edit distance.
• I2. Build two complementary feature views from the same decomposition, one keeping the core characters and the inverse one keeping only the residue, so each view's neighbours supply evidence for the other lexicon.
• I3. Raw probabilities fell below the baseline, while log scaling and morpheme-length adjustment added roughly 11 points, so the way evidence is aggregated can matter as much as the model itself.
• I4. A count-only baseline, which uses the whole vocabulary as one cluster with uniform proximity, isolates exactly what the learned clusters contribute.
• I5. Enumerate every candidate root–pattern analysis of a word and pick the one with the best summed lexicon scores, which keeps each decision transparent.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): This could cluster unknown or rare English word forms into derivational families, giving the lemmatizer candidate lemmas it can explain.
• I2 (tokens): Contrasting stem-view and affix-view neighbours could help induce an English affix inventory (un-, -ness, -ize) to back the morphology analyser.
• I3 (learn): When combining confidences from several witness parsers into silver data, use log-scaled, length-normalised scores rather than summing raw probabilities.
• I4 (learn): Every learned component could be paired with such a frequency-only control as a negative check, so gains are attributable to the method.
• I5 (tokens): Morphological analyses could be logged with their competing candidates and score breakdowns, supporting Mova's requirement that every step be explainable.