Elamur Research⌕
☰
home / papers / 2021 / Country-level Arabic Dialect Identification using RNNs with and without Linguistic Features

Country-level Arabic Dialect Identification using RNNs with and without Linguistic Features

Paper · 2021 · Elsayed Issa, Mohammed AlShakhori, Reda Al-Bahrani et al.

Summary

The paper addresses the NADI 2021 Subtask 1.2: assigning Arabic tweets to one of 21 country-level dialects. This is hard because MSA and the regional varieties share one script, and tweets usually omit short-vowel diacritics, so many written strings look the same across dialects. The authors use only the official NADI Twitter corpus, cleaned of non-Arabic tokens, emojis and links. They compare two models. The first is a word-level LSTM over pretrained Mazajak CBOW embeddings (Aravec was also tried). The second is a hybrid that adds a branch encoding 56 hand-crafted binary dialect markers as learned dense embeddings. Against expectations, the hand-crafted features lowered macro F1 on the development set, so the plain LSTM became the submitted system. The most useful part of the paper is its error analysis. It names four causes: dialect-neutral wording, topics shared across the Arab world, users whose dialect differs from their country label, and markers that are regional rather than national. The work shows that a small set of sparse hand-picked cues can add noise on top of pretrained representations instead of adding signal.

Key points

  • On the dev set, Model 1 (LSTM with Mazajak CBOW) scored 41.36 accuracy and 22.10 macro F1. Model 2 (LSTM plus features) scored 37.82 accuracy and 18.60 macro F1.
  • Model 1 settings:
    • 300-unit LSTM with dropout 0.3 and recurrent dropout 0.2.
    • Input capped at 80 tokens, vocabulary of 50,000, embedding size 300, batch size 64.
    • Early stopping with patience 2, using 10% of the training set held out for validation.
  • The Mazajak embeddings were trained on about 100M tweets and slightly beat Aravec.
  • The feature branch works as follows:
    • 56 binary lexical markers (for example demonstratives and degree words) are each mapped to a dense embedding.
    • These pass through two 100-unit ReLU layers and an element-wise multiplication.
    • The result is concatenated with the LSTM output.
    • Features marking the absence of a cue (NOT X) were also explored.
  • The failure analysis lists five causes: bivalent linguistic units, "global genre" topics such as football, participant mobility, cues shared across a region or meaning different things in different dialects, and low feature coverage.

Interesting ideas

  • I1. The authors tested whether hand-built cues add anything once a strong pretrained encoder is in place, and they reported the negative result honestly instead of assuming linguistic knowledge must help.
  • I2. Some training items carry almost no signal for their label because the text is valid in many varieties, so the authors treat them as inherently ambiguous rather than as model errors.
  • I3. A word can be a strong cue in one variety and mean something unrelated in another, for example "then" in Gulf dialects versus "child" in Egyptian. Lexical cues therefore need to be conditioned on the variety.
  • I4. Presence-only binary cues are turned into learned embeddings and fused with a sequence encoder, and absence cues are considered as well. This is a cheap, extensible way to inject symbolic knowledge into a neural model.
  • I5. The paper links weak cue usefulness to low corpus coverage relative to the number of classes, which suggests checking coverage before trusting sparse rules.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Before adding a hand-written rule or feature to a learned component, run a controlled ablation against the bare model and keep the addition only if it beats that baseline.
  • I2 (learn): When building silver data from witness parsers, explicitly mark sentences where readings legitimately diverge, so they are not used as gold or counted as disagreements to repair.
  • I3 (tree): During UD dialect conversion and treebank merging, keep lemma and sense mappings specific to each source variety instead of applying a single global lexicon.
  • I4 (tokens): Morphological or lexicon flags could be fed to the tagger as learned embeddings alongside word vectors, with each flag remaining traceable for explanations.
  • I5 (learn): Self-induced repair rules could be required to reach a minimum firing frequency on held-out text before being promoted, which would screen out rules that are rarely triggered.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Elsayed Issa, Mohammed AlShakhori, Reda Al-Bahrani and Gus Hahn-Powell. 2021. Country-level Arabic Dialect Identification using RNNs with and without Linguistic Features. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{issa2021country,
  title = {{Country-level Arabic Dialect Identification using RNNs with and without Linguistic Features}},
  author = {Elsayed Issa, Mohammed AlShakhori, Reda Al-Bahrani and Gus Hahn-Powell},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.32/}
}

References

29 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. You tweet what you speak: A city-level dataset of arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
  2. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop (WANLP 2020), Barcelona, Spain. ↗
  3. Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017. Language modeling with gated convolutional networks. In International conference on machine learning, pages 933–941. PMLR. ↗
  4. Anna De Fina. 2000. Orientation in immigrant narratives: The role of ethnicity in the identification of characters. Discourse Studies, 2(2):131–157. ↗
  5. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers), pages 49–54.
  6. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop (WANLP 2021). ↗
  7. Mohamed Elaraby and Muhammad Abdul-Mageed. 2018. Deep models for arabic dialect identification on benchmarked data. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 263–274.
  8. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics.
  9. Heba Elfardy and Mona Diab. 2013. Sentence level dialect identification in arabic. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 456–461.
  10. Oscar Araque, Ignacio Corcuera-Platas, J Fernando Sánchez-Rada, and Carlos A Iglesias. 2017. Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Systems with Applications, 77:236–246. ↗
  11. Charles A Ferguson. 1959. 15(2):325–340. find
  12. Reem Bassiouney. 2009. Arabic sociolinguistics: Topics in diglossia, gender. Identity, and Politics. Georgetown University Press. find
  13. Kara Becker. 2009. /r/and the construction of place identity on new york city’s lower east side 1. Journal of Sociolinguistics, 13(5):634–658. find
  14. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. The journal of machine learning research, 3:1137–1155.
  15. Sarah Bunin Benor. 2010. Ethnolinguistic repertoire: Shifting the analytic focus in language and ethnicity 1. Journal of Sociolinguistics, 14(2):159–183. find
  16. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207.
  17. Mary Bucholtz. 2010. White kids: Language, race, and styles of youth identity. Cambridge University Press. find
  18. Mary Bucholtz and Kira Hall. 2005. Identity and interaction: A sociocultural linguistic approach. Discourse studies, 7(4-5):585–614. ↗
  19. François Chollet et al. 2015. keras.io. Keras. https://Diglossia. ↗
  20. WORD, Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9.
  21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. ↗
  22. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431, Valencia, Spain. Association for Computational Linguistics. find
  23. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410. ↗
  24. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of the 12th language resources and evaluation conference, pages 7022–7032.
  25. Natalie Schilling-Estes. 2004. Constructing ethnicity in interaction. Journal of Sociolinguistics, 8(2):163– 195. ↗
  26. Aliaksei Severyn and Alessandro Moschitti. 2015. Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 959–962. find
  27. Abu Bakr Soliman, Kareem Eissa, and Samhaa R El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  28. Kathryn A Woolard. 1998. Simultaneity and bivalency as strategies in bilingualism. Journal of linguistic anthropology, 8(1):3–29. ↗
  29. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/issa-2021-country-level-arabic-dialect-identification/