📄 Annotating and Learning Morphological Segmentation of Egyptian Colloquial Arabic

Summary

Tools built for Modern Standard Arabic (MSA) perform poorly on Egyptian Colloquial Arabic (ECA), which is written online with non-standard spelling, merged clitics, assimilated morpheme boundaries and inconsistent hamza and vowel spelling. The authors built a segmentation resource and a baseline segmenter for this dialect. Two trained native-speaker annotators marked morpheme boundaries with "+" in political comments and jokes from masrawy.com. Their agreement was very high.

The segmenter treats each letter as a binary decision: either a boundary follows it or it does not. It uses the Timbl memory-based learner and only the characters inside the word as context. On ECA test data it reaches about 92% exact-match word accuracy when ECA and normalized ATB data are combined. MADA scores 81% and an ATB-only segmenter 82%.

Most remaining errors come from letters that colloquial spelling makes ambiguous. Orthographic normalization reduces sparseness but adds ambiguity. The work shows that a small dialect-specific annotated corpus can beat a strong standard-language system.

Key points

• Corpus: 320 training comments (20,022 words) and 36 test comments (2,445 words). Inter-annotator Kappa was 99.27%, measured on a 2,899-word sub-corpus. About 45% of training words have a single segment.
• Model: Timbl IB1 with overlap similarity, gain-ratio feature weights and k=1. Features are the five characters before and the five after the focus letter.
• Word exact-match accuracy:
  • MADA: 81.48
  • ATB-trained MSA segmenter: 81.52
  • ECA: 88.50
  • ECA+norm: 89.20
  • ECA+MSA+norm: 91.90 (letter accuracy 97.80)
• 189 of MADA's 468 error words (40%) received NO-ANALYSIS. Without those words, MADA's accuracy would be 88.16%.
• Lexical coverage matters most. The OOV rate and accuracy have a Pearson correlation of −0.9885, and in-vocabulary accuracy stays around 98–99%. Final Y causes 10.6% of missed boundaries, and h causes 28.33% of over-segmentation errors.

Interesting ideas

• I1. Framing segmentation as a yes/no boundary decision for each character, with a tiny window inside the word, makes a segmenter that is simple, fast and easy to inspect.
• I2. Memory-based learning with k=1 answers by pointing to the closest stored training instance, so every decision comes with a concrete precedent.
• I3. Score a word as correct only when every boundary in it is right, because one segmentation slip spreads to tagging and parsing.
• I4. Normalization involves a trade-off: merging spelling variants reduces sparseness but erases distinctions and creates new ambiguity, and known-word accuracy actually fell after normalizing.
• I5. Splitting results into in-vocabulary and out-of-vocabulary words, and correlating the OOV rate with accuracy, shows whether gains come from generalization or from coverage.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): A per-character boundary classifier could split English clitics, contractions and hyphenated or noisy tokens, and each split could be traced to the stored example that triggered it.
• I2 (tokens): The tagger or lemmatizer could cite the nearest memorized example as its explanation, which fits the requirement that every step be explainable.
• I3 (learn): Evaluation could report whole-unit exact match (whole token, whole tree, whole frame) next to partial-credit scores, so small local errors that ruin later stages are not hidden.
• I4 (tokens): Normalized and original forms should be kept side by side, with the original form available to later stages when the normalized one becomes ambiguous.
• I5 (learn): Every evaluation could report IV/OOV splits, so that it is clear whether new silver data helps through real learning or only by adding lexicon entries.