📄 AIDA: Identifying Code Switching in Informal Arabic Text

Summary

The paper addresses token-level detection of code switching between Modern Standard Arabic (MSA) and Egyptian dialectal Arabic (EDA) in informal Twitter text. It applies an updated version of the authors' AIDA system to the MSA-DA data of the EMNLP 2014 shared task on code-switched language identification. The authors did not compete officially because colleagues organized the task.

AIDA chains several components:

• light cleanup or clitic tokenization;
• a 5-gram language model over tagged tokens, decoded with Viterbi;
• MADAMIRA morphological analysis, including which underlying analyzer supplied the glosses;
• the ANERGazet name lists.

A rule-based combiner merges these signals into one of six labels.

Training mixes a small manually annotated tweet set with millions of weakly labeled forum words. On token-level weighted F-score, AIDA reaches 93.6%, 77.7% and 80.1% on the two test sets and the surprise-genre set. Averaged over all three, it slightly leads the other systems. It is also the best participant at recognizing named entities. Its tweet-level switch detection is weak. The work shows that a transparent pipeline can be competitive with lexical, morphological and gazetteer evidence plus weak supervision.

Key points

• The tag set has six labels: lang1 (MSA), lang2 (DA), mixed, other, ambig and NE. The NE label is assigned from ANERGazet (1,545 locations, 2,100 people, 318 organizations) or from MADAMIRA's proper-noun POS.
• The training data has two sources:
  • shared-task Twitter data (STT, 119,326 words), split 90/10 into train and dev;
  • 8 million weakly labeled web-log words (WLT), half MSA and half EDA, with labels taken from the forum's dialect.
• To match the weak data, STT labels were collapsed so that each tweet gets its majority tag (lang1 or lang2). STT named entities were added to the gazetteer.
• The tokenized setup (D3 clitic segmentation via MADAMIRA) beat the surface-form setup in every experiment. The chosen configuration, Tokenized-2, adds two copies of STT-Tr to WLT and reached 77.6% weighted token F on dev.
• Test-set results:
  • token-level weighted F of 93.6%, 77.7% and 80.1%;
  • tweet-level F of 4.4%, 36% and 27.7%;
  • averaged token F of 86.8%, against 86.4% for CMU.
• Typical errors:
  • lang1 words tagged lang2;
  • named entities that double as ordinary words, such as a show title also meaning "clearly";
  • mixed words that MADAMIRA failed to segment.

Interesting ideas

• I1. An ordered rule combiner decides the final tag in a fixed cascade: punctuation/numbers, then gazetteer/proper noun, then LM tag, then morpheme disagreement, then analyzer provenance. Each decision can therefore be traced to a single rule.
• I2. For out-of-vocabulary words, the system uses the source lexicon of the morphological analysis as evidence: an analysis from the standard lexicon (SAMA) means MSA, and one from the dialect lexicon (CALIMA) means dialect.
• I3. A word's tag is copied onto all its clitics. A word whose morphemes receive different tags is then labeled "mixed", so hybrid forms are detected from sub-word disagreement.
• I4. Gold data was relabeled to match the coarser scheme of a much larger weakly supervised corpus, and repeated a few times to boost its weight. The best repeat count was chosen on dev.
• I5. Combining gazetteer lookup with the analyzer's proper-noun POS proved the most robust entity detector in the task. The remaining errors were mostly names that are also common words.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): A similar cascade could resolve token categories such as foreign word, name or number, logging which rule fired so the tagger's output stays explainable.
• I2 (tokens): Recording whether a lemma came from a core dictionary or from a slang/informal lexicon would give a cheap, inspectable register feature for unknown or nonstandard words.
• I3 (tree): When splitting UD multiword tokens such as contractions, Mova could keep per-part labels and flag internally inconsistent tokens as candidates for review.
• I4 (learn): When mixing a small gold treebank with large silver data, Mova could harmonize the label schemes first and tune the gold upsampling factor against a held-out set.
• I5 (world): Entity creation could require agreement between a Wikidata-style name list and the tagger's PROPN. Name/common-word homographs could be flagged explicitly for contextual checking.