📄 POS Tagging of Dialectal Arabic: A Minimally Supervised Approach

Summary

The paper tackles part-of-speech tagging for Egyptian Colloquial Arabic (ECA). This spoken dialect has very little annotated text, no fixed spelling and almost no dedicated tools. The authors avoid building dialect-specific resources. They start from raw text and the MSA-oriented Buckwalter stemmer: the stemmer's possible analyses for each word form a tag lexicon, and a trigram HMM (built in GMTK) is then trained with EM on unlabeled CallHome ECA transcripts. On top of this baseline they add three things:

• affix features;
• a tighter lexicon for words the stemmer cannot analyse;
• cross-dialect data sharing with the Levantine (LCA) corpus.

The most effective sharing scheme trains a single tag-sequence model jointly on both dialects but keeps a separate word-emission model for each one, with ECA data upweighted. On the ECA evaluation set, accuracy rises from 58.47% to 68.48%. This is about a 10-point gain without any hand-built dialect lexicon, and it shows that a closely related variety's resources can support a low-resource one.

Key points

• Resources: CallHome ECA (telephone speech, romanized and then converted automatically to script), the LDC Levantine corpus (raw text only), the LDC MSA Treebank parts 1–3, and the Buckwalter stemmer. All tags were collapsed into a coarse set of 17 categories following Bies (2003).
• Evaluation scope: only the roughly 75% of ECA tokens that have a single lexicon tag count in scoring. The ambiguous ~25% are left out.
• Baselines compared:
  • System I: a supervised MSA tagger.
  • System II: a supervised ECA tagger.
  • System III: an unsupervised tagger with an expert-derived lexicon.
  • System IV: an unsupervised tagger with a Buckwalter-derived lexicon.
• Dev results: JointTrain(2:1)+affix reached 70.88%. Simply pooling ECA and LCA (CombineData) with affixes gave only 61.78%. Counting a guess as correct whenever it falls in an ambiguous word's tag set would lift accuracy to an optimistic 77.18%.
• Eval results: System IV scored 58.47%, adding affix features and the constrained lexicon gave 66.61%, and joint training with affixes gave 68.48%. These gains are significant at the 0.005 level.

Interesting ideas

• I1. The tagger can switch between per-source lexical models depending on where a word came from, while all sources share one contextual (tag-transition) model; upweighting the target corpus pulls the shared model toward it.
• I2. An analyzer built for a related standard variety can supply candidate tag sets, and EM on unlabeled text then picks the right tag; unanalyzable words get a narrowed candidate set instead of the full tagset.
• I3. A naive "pool all the data" system with the same features serves as a control, which shows the gain comes from the modeling technique rather than from simply having more data.
• I4. Tokens whose gold label is itself ambiguous are excluded from strict scoring, and a separate optimistic upper bound counts any tag inside the allowed set as correct.
• I5. Before adding data from a related source, measure how much it raises n-gram coverage of the target development set, as a cheap check that the data could help at all.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The en tagger could share one tag-transition model across genres such as fables, school texts and word problems, with a separate emission model per genre, so a small genre borrows syntax statistics without inheriting another genre's vocabulary.
• I2 (tokens): Mova's morphology could provide the allowed tags for words in non-standard or informal English, with EM settling the choice and only open-class tags offered for unknown words.
• I3 (learn): Every new silver-data or augmentation method should be reported against an equal-data naive-pooling baseline, as a built-in negative control.
• I4 (learn): Where witness parsers or treebank conventions legitimately disagree, Mova could report both a strict score and a set-membership upper bound, rather than hiding the ambiguity.
• I5 (tree): Before mixing a converted treebank of another English variety into gparse training, compare word and n-gram coverage of Mova's target texts to predict whether it is worth the cost.