📄 Arabic Part of Speech Tagging

Summary

The paper tackles POS tagging of Arabic, a language where one orthographic word can contain clitics, inflections and a stem. Earlier work relied on the gold segmentation in the Penn Arabic Treebank (ATB), which real text does not provide. The authors restored newswire-like words from two non-overlapping ATB sections (about 500,000 words, unvocalized), reattaching clitics and dropping pro-drop null subjects. They kept the full ATB tagset and compared two setups, both built on memory-based learning. In the first, MBT assigns complex whole-word tags with no segmentation. In the second, a TiMBL segmenter splits words and MBT then tags each segment. Accuracy was measured per word using 5-fold cross validation. Whole-word tagging scored slightly higher overall (93.93% vs 93.41%) and did best on known words. The segment pipeline handled unknown words better. A hybrid that uses each method where it is strong reached 94.37%, close to the 94.91% obtained with gold segmentation. This shows that competitive Arabic tagging is possible without a morphological analyzer.

Key points

• Data: ATB parts P1V3 and P3V1, chosen because they share no duplicate sentences. The tagset has 993 complex whole-word tags versus 139 segment tags.
• Known/unknown word accuracy:
  • whole words: 96.60% / 65.50%
  • automatic segmentation: 95.55% / 70.50%
  • gold segmentation: 95.90% / 84.25%
  • baseline: 85.57% / 3.67%
• The hybrid tags a word whole if it was seen in training and through segments otherwise. It reached 94.37% word accuracy.
• Error analysis:
  • NOUN/ADJ confusions account for over 15% of errors.
  • Proper nouns tagged as NOUN account for 9.1%.
  • Segmentation mistakes cause 28% of errors, and the segmenter splits too often (2,201 over-segmented words vs 1,089 under-segmented).
• The authors mapped their output to the 15-tag reduced set of Habash and Rambow (2005). Under that mapping, the whole-word tagger gets 99.33% tokenization and 96.41% POS accuracy, compared with their 98.1/96.5 (on different data splits).

Interesting ideas

• I1. Pick the analysis route per word by whether it was seen: memorized whole-word tags for seen words, compositional segment analysis for unseen ones.
• I2. Score the pipeline on the units real input actually has (whole words), even when internal processing works on smaller pieces, so preprocessing errors are not hidden.
• I3. Run the same split three ways (gold preprocessing, automatic preprocessing, no preprocessing) to separate the upper bound from the cost of each pipeline stage.
• I4. Classify errors by their structural cause, such as too many or too few segments, rather than only by tag confusion. This exposes systematic biases like over-splitting.
• I5. When comparing to prior work with different conventions, deterministically map your richer output down to their coarser scheme and state each mapping decision.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The en tagger could look up full-form tags for in-vocabulary words and fall back to affix/morphology-driven tagging for OOV words, logging which path was taken so the choice stays explainable.
• I2 (learn): Evaluation should report accuracy on raw-text tokens, so that tokenizer splitting mistakes are counted against downstream tagging and parsing instead of being masked by gold tokenization.
• I3 (tree): Parsing gparse and the perceptron parser on gold tokens/tags versus predicted ones would measure exactly how much LAS each upstream error type costs.
• I4 (learn): Self-induced repair rules could be mined from error buckets like "over-split contraction" or "missed clitic", which give a clear, explainable target for each rule.
• I5 (tree): UD dialect conversion between treebanks could follow this practice by recording every lossy merge (like POSS→PRO here), so cross-treebank scores remain interpretable.