📄 Using Twitter to Collect a Multi-Dialectal Corpus of Arabic

Summary

Arabic speakers mostly use regional dialects in daily life, while most NLP resources cover Modern Standard Arabic (MSA). Dialect classifiers trained on n-grams depend heavily on how well their training data matches the target genre. Mubarak and Darwish instead build country-level dialect corpora from Twitter. They combine two signals. The first is the location users declare in their profiles, mapped to an Arab country with GeoNames plus manual revision. The second is a hand-judged list of dialectal n-grams drawn from the Arabic Online Commentary Dataset (AOCD), which decides whether a tweet is dialectal at all. From 175M Arabic tweets, this pipeline keeps 6.5M dialectal tweets, most of them from Saudi Arabia, Egypt and Kuwait. Native speakers judged samples through crowdsourcing. Country assignment is highly accurate for Saudi Arabia and Egypt, but much weaker for Syria and Algeria. The authors also show that most supposedly Egyptian dialect words appear in other dialects too. This argues against collecting per-country corpora from lexical cues alone, and it questions the usual five-region grouping of dialects.

Key points

• Collection and location filtering:
  • In March 2014 the authors collected 175M tweets with the query lang:ar, roughly 5.6M per day.
  • About 70% of tweets (123M) had a user-declared location.
  • The top 10K distinct locations accounted for about 75% of those tweets.
  • Around 3,500 locations were mapped to a single country, leaving 62M tweets with an unambiguous country.
• Dialectal n-gram list:
  • Roughly 45,000 n-grams from AOCD, each seen at least 3 times, were judged by one native speaker.
  • About 2,500 of them were dialectal: 54% unigrams, 39% bigrams and 7% trigrams.
  • A tweet containing any one of these n-grams was treated as dialectal.
• Filtered corpus:
  • The filter kept 6.5M tweets, about 3.7% of the original collection.
  • Saudi Arabia contributed 61%, Egypt 13%, Kuwait 11%, the UAE 5% and Qatar 2%.
• Evaluation:
  • The authors sampled 100 tweets per dialect and collected 3 CrowdFlower judgments for each from people in the matching country.
  • 20 challenge questions screened the judges.
  • Accuracy was 95% for Saudi, 94% for Egyptian, 82% for Iraqi, 75% for Lebanese, 66% for Syrian and 60% for Algerian.
  • No judges could be found for Qatar or Bahrain.
• Contamination of MSA corpora: Supposedly MSA corpora contain many dialectal words, through interview transcripts and quotations. This holds for the 114M-token Aljazeera corpus and for Arabic GigaWord, where kdh alone occurs 2,574 times.

Interesting ideas

• I1. The country label comes from user metadata, while a separate lexical filter decides whether a tweet is dialectal; combining two weak, independent signals yields a usable labelled corpus without training a classifier.
• I2. A reference corpus assumed to be "clean" (MSA news) turned out to hold dialect inside quotes and interview transcripts, which motivated manual judging instead of frequency contrast.
• I3. Location strings follow a heavy-tailed distribution, so manually curating only the most frequent ones covered most of the data cheaply.
• I4. Locations that do not point to a single country, such as "Arabian Peninsula", were dropped rather than guessed, trading volume for precision.
• I5. Evaluation used judges from the target community, screened by hidden challenge questions, and reported accuracy per class rather than as one average.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver examples could be kept only when two unrelated cues agree, for example a parser vote and a lexical or metadata cue, which reduces correlated errors among witness parsers.
• I2 (tree): Treebanks and silver corpora should be checked for embedded quoted or dialogue registers, which can be tagged separately so they do not skew parser statistics for narrative text.
• I3 (world): Hand-checking the links from the most frequent entity mentions to Wikidata or WordNet would cover most real text at low cost, leaving the long tail to automatic linking.
• I4 (world): The entity resolver should abstain on mentions that are ambiguous in principle and record the abstention as an explainable state rather than forcing a link.
• I5 (learn): LLM teacher tiers could be audited with planted gold items and scored per construction type, which would expose weak categories that an overall score hides.