📄 Adapting Standard Open-Source Resources To Tagging A Morphologically Rich Language: A Case Study With Arabic

Summary

The paper asks whether usable part-of-speech tagging for Modern Standard Arabic (MSA) can be assembled from freely available components, using a model trained only on Classical/Quranic Arabic. That training text lacks modern vocabulary, differs somewhat in grammar, and carries full diacritics, while MSA normally omits short vowels. The author surveys open-source Arabic analysers and taggers and finds few of them, with mismatched encodings, tokenisation schemes and tag granularities.

The first plan, Alkhalil plus the Stanford tagger, failed. Stanford offered no way to handle unseen Arabic words, plug in an analyser, or expose tag confidence. The working prototype instead pairs BAMA 1.2 with TreeTagger trained on the Quranic Arabic Corpus. The analyser's candidate tags, mapped to the Quranic tag set, restrict what the tagger may choose for each word.

On a small hand-annotated BBC Arabic sample, the combination reaches about 73%. The analyser alone already scores 69%, so the tagger adds little. The study shows that classical text alone is inadequate training data for MSA and documents concrete integration obstacles.

Key points

• Training data: Quranic Arabic Corpus (Dukes, 2009), 77,430 words, fully diacritized, split on whitespace only. No freely tagged MSA corpus was obtainable.
• Test data: a 66-word extract from BBC Arabic, tagged by hand by an Arabic speaker. The baseline labels every word N (44% on MSA, 36% on QA).
• In-domain accuracy on QA: Stanford 98%, TreeTagger 96%. On MSA both drop below the all-noun baseline: Stanford 39%, TreeTagger 35%.
• BAMA by itself reaches 69% on MSA. Analyser plus TreeTagger with three appended candidate tags reaches about 73%, both with and without frequency probabilities. PRON recall is only 25% and ADJ scores 0.
• Tag mapping: ABBREV and INTERJ became N. ADV became T (time adverb) because T was more frequent than LOC in training (1115 vs 656). All morphological features were collapsed to the bare category.

Interesting ideas

• I1. Analyser-constrained tagging: a morphological analyser supplies each token's admissible tags, optionally with probabilities, and the statistical tagger only chooses among them. This keeps lexical knowledge explicit and checkable.
• I2. Majority-class fallback for unmappable distinctions: when the source scheme cannot make a split the target scheme needs (ADV vs T/LOC), map to the target tag that is most frequent in training, and document the choice.
• I3. Component ablation: score the analyser alone, the tagger alone and the combination, which revealed that the tagger contributed only about four points over the analyser.
• I4. A trivial baseline as a negative control: an all-noun tagger beat both trained taggers out of domain, exposing how badly domain shift hurt them.
• I5. Selecting tools by integration hooks: the Stanford tagger was dropped because it lacked unknown-word handling, analyser input and confidence output.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): the en tagger could accept per-token candidate sets from the morphology/lexicon and record which candidates it rejected, which makes every tag decision explainable.
• I2 (tree): UD dialect conversion could apply the same rule for lossy label mappings, logging each forced choice so that later repair rules can revisit it.
• I3 (learn): evaluations should report each pipeline stage's marginal gain, so that a stage adding almost nothing is detected rather than assumed useful.
• I4 (learn): every out-of-domain test (e.g., new genres) should include a dumb baseline, so that a trained model falling below it raises an alarm.
• I5 (tokens): when choosing or designing components, require constraint input and per-decision confidence as hard criteria, since these are what make explainable pipelines composable.