📄 A Character Level Convolutional BiLSTM for Arabic Dialect Identification

Summary

The paper addresses fine-grained Arabic dialect identification at the level of Twitter users. Each user must be assigned one of 21 countries, which is harder than separating a few broad dialect groups because neighbouring dialects look very similar in writing. The CU-RAISA team built this system for MADAR shared task 2 (2019). They crawled the organizers' tweet IDs, removed URLs and mentions, and kept a character-only vocabulary. The model is a character-level network: parallel convolutions of many widths feed a BiLSTM, which ends in a softmax over the countries. The authors compared two ways of framing the input. One classifies each tweet separately, and the other joins all of a user's tweets into one long sequence. Joining the tweets worked far better, both for the neural models and for n-gram baselines. On the official test set the system placed 4th, with 61.5% F1-macro and 72.6% accuracy. The work shows that for author-level labels, giving the model the whole body of a user's text matters more than per-message tricks.

Key points

• Data came from the MADAR Twitter user dialect task. The official split was 2180 training users (217,593 tweets), 300 development users and 500 test users. Some timelines could not be retrieved, so the usable sets shrank to 2032, 281 and 463 users.
• Preprocessing stripped URLs and user mentions. Characters seen fewer than 20 times were dropped, leaving a vocabulary of 2377 characters.
• Architecture and training:
  • The model uses 50-dimensional randomly initialized character embeddings.
  • Convolutions have filter widths from 1 to 13 and between 10 and 150 filters, followed by a time-distributed layer.
  • The BiLSTM has 256 hidden units, and the softmax covers 21 classes.
  • Training used Adam for 40 epochs, with dropout 0.2 on the embeddings and L2 regularization of 0.002.
• Two input framings were tested. Per-tweet inputs were capped at 140 characters, while concatenated user inputs reached up to 12,000 characters.
• Development accuracy for C-BiLSTM rose from 51.7% on single tweets to 71.92% on concatenated tweets, and F1-macro rose from 42.3 to 62.21. Logistic regression and Multinomial Naive Bayes on 1–11 n-grams stayed below 47% even with concatenation.

Interesting ideas

• I1. Feed the model input at the same granularity as the label: joining all of one author's messages beat voting over per-message predictions, because weak cues add up across texts.
• I2. A plain frequency cutoff on the character inventory keeps the vocabulary compact while preserving rare-but-real symbols that word models would miss.
• I3. Users with no retrievable data still had to receive a label, so the system gave them the most frequent class (Saudi Arabia); this makes the fallback an explicit, measurable part of the system.
• I4. Running the same n-gram baselines under both input framings separates the gain from the framing itself from the gain from the neural architecture.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Document-wide judgments, such as irony or a narrator's stance, could be computed from all relevant clauses together rather than by voting over sentence-level calls.
• I2 (tokens): A character-level fallback with a frequency-pruned inventory could help the tagger and lemmatizer cope with misspellings, social-media spellings and unknown words.
• I3 (learn): Evaluations could report what share of the score comes from a majority-class fallback, using it as a negative control that real reasoning must clearly beat.
• I4 (learn): When a new component is added, rerunning simple baselines under the changed setup would show whether the improvement comes from the method or merely from richer input.