📄 Multi-Dialect Arabic POS Tagging: A CRF Approach

Summary

The paper addresses the lack of part-of-speech resources for dialectal Arabic on social media. The authors labeled 350 tweets apiece from four major dialects, namely Gulf, Egyptian, Maghrebi and Levantine. They used an extended Farasa tagset with extra tags for dialect particles and tweet-specific tokens. Annotation was applied to the original, unnormalized spelling, with each word split into clitics so that every segment receives exactly one tag.

Using a CRF sequence labeler, they compared three feature types: clitic n-grams, clitic metatypes and stem templates. They also ran three training regimes: per-dialect, cross-dialect and joint. A size-matched MSA sample from ATB served as an extra variety for comparison.

The main result is a single joint model that tags all four dialects with an average accuracy of 89.3%. This is on par with dialect-specific taggers. It means a pipeline does not need an unreliable dialect-identification step to choose a tagger. The dataset and its splits are released publicly, making it the first openly available benchmark of this kind.

Key points

• Dataset size: 350 tweets per dialect, giving about 7,481 (EGY), 7,221 (LEV), 6,767 (GLF) and 6,400 (MGR) words. Each dialect has 5-fold splits with a 70/10/20 train/dev/test ratio.
• Tagset: the 18 Farasa MSA tags plus 2 dialect tags (PROG PART, NEG PART) and 4 tweet tags (HASH, EMOT, MENTION, URL). Hashtags, mentions, emoticons and URLs are kept as single unsegmented words.
• Model: CRF++ with L2 regularization and C=10, assuming gold segmentation. Features are clitic n-grams (up to 4-grams), metatypes, and stem templates obtained from Farasa.
• Baselines and evaluation: a 350-sentence ATB sample (7,385 words) was used as an MSA baseline. Accuracy is measured per word, which is stricter than per clitic.
• Findings from analysis:
  • Compared with MSA, dialects show more verbs, pronouns and particles, and fewer nouns, adjectives, prepositions, numbers and definite articles.
  • The most frequent error is ADJ→NOUN, at about 21% of errors on average.
  • Brown clusters are proposed as future work.

Interesting ideas

• I1. Instead of a classifier that routes text to variety-specific models, train one model on pooled data from all varieties, which avoids errors cascading from a weak routing step.
• I2. Segment words so that the number of pieces matches the number of tags, and keep the writer's original spelling rather than normalizing it.
• I3. Mark token, word and sentence boundaries explicitly in the data format, so the clitic level and word level never get confused.
• I4. Report accuracy at the word level, where a word counts as correct only if all its parts are tagged correctly, because per-piece scores look inflated.
• I5. Train on one variety and test on another, building a full matrix of results, and compare tag distributions against a size-matched formal sample. This helps separate genre effects from genuine linguistic differences.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): train the parser jointly on several English UD treebanks of different genres (news, web, social) instead of detecting the genre first and choosing a specialized model.
• I2 (tokens): split contractions and fused forms (e.g. "gonna", "don't") into pieces that each carry one UPOS while preserving the surface string, so every tag can be traced back to raw text.
• I3 (tokens): keep CoNLL-U multiword-token ranges alongside syntactic words so that downstream explanations can cite both levels.
• I4 (learn): evaluate taggers and parsers on whole-word or whole-span correctness as well as per-token scores, to avoid overestimating quality.
• I5 (learn): build cross-genre train/test matrices and compare tag distributions as a sanity check showing whether silver data drifts away from the gold treebanks.