📄 Curriculum Learning and Pseudo-Labeling Improve the Generalization of Multi-Label Arabic Dialect Identification Models

Summary

Arabic Dialect Identification has usually been treated as assigning one country label per sentence. In reality, many utterances sound natural in several dialects. The multi-label variant (MLADI) still has no large multi-label training resource: only small evaluation sets carry multiple labels. The authors trained binary dialect acceptability classifiers on single-label NADI 2020, 2021 and 2023 data and analyzed their training dynamics. This showed that the core obstacle is negative sampling: many tweets from other countries labeled as negatives are actually acceptable in the target dialect.

To address this, they built a pseudo-labeled multi-label training set. It merges GPT-4o annotations with predictions from 18 binary classifiers, and the merging rule depends on each sentence's ALDi score. On this data they trained LahjatBERT, a BERT-based multi-label model, with and without curriculum learning (ordered by ALDi or by label cardinality). The ALDi-curriculum model reaches about 0.69 macro F1 on the MLADI leaderboard, compared with 0.55 for the best earlier system. This shows that single-label corpora can be turned into useful multi-label supervision.

Key points

• Training data: NADI 2020, 2021 and 2023 tweets, geo-labeled for 18 countries. Evaluation used the MLADI development set (120 sentences, 8 dialects) and test set (1,000 sentences, 11 dialects).
• Diagnosing label noise: Training dynamics, following Swayamdipta et al., revealed that a substantial share of "negative" samples are acceptable to native speakers. Flagged cases still need manual checking.
• Hybrid pseudo-labels: The binary classifiers' conservative predictions are used at the low and high ends of the ALDi scale. GPT-4o's richer labels are used in the mid-range.
• Test-set macro F1:
  • LahjatBERT + ALDi curriculum: 69.0 (best)
  • LahjatBERT without curriculum: 68.0
  • Aya-32B: 54.5
  • Elyadata (top NADI2024 system): 52.4
  • NADI 2024 baseline: 47.0
• Curriculum trade-off: Both curricula increase recall at some cost to precision; the cardinality curriculum reaches 81.0 recall on test. Only tweet IDs and derived labels are released.