Elamur Research⌕
☰
home / papers / 2018 / Arabic Dialect Identification in the Context of Bivalency and Code-Switching

Arabic Dialect Identification in the Context of Bivalency and Code-Switching

Paper · 2018 · Mahmoud El-Haj, Paul Rayson and Mariam Aboelezz

Summary

The paper addresses automatic identification of written Arabic dialects. This is hard because many words have the same meaning across varieties (bivalency), and writers often mix dialect with Standard Arabic (code-switching). The authors build a five-class dataset covering Egyptian, Gulf, Levantine and North African dialects plus MSA. Most of it comes from the crowd-annotated Arabic Commentary Dataset (AOC), with North African forum text from the Tunisiya corpus.

They argue that word n-grams alone generalize poorly with small corpora. They therefore add three kinds of features:

  • Grammatical: POS tags from an MSA-trained Stanford tagger and counts of function words.
  • Stylistic: type-token ratio and the OSMAN readability metric.
  • Subtractive Bivalency Profiling (SBP): a new method, inspired by corpus linguistics, that explicitly handles words shared between dialects.

Four standard classifiers are compared. SBP is the strongest single feature group, and combining it with the other features helps a little. On a separate unseen test set, the best accuracy is about 66%, compared with roughly 42% for n-grams. This shows that overlap-aware features transfer better than surface lexical patterns.

Key points

  • The training data has 16,494 sentences (355,069 words) across EGY, GLF, LAV, MSA and NOR. A separate unseen test set has 7,073 sentences (138,878 words). The dataset and code are freely released on GitHub (drelhaj/ArabicDialects).
  • Only AOC sentences that annotators judged "mostly dialect" were kept. NOR was taken from the Internet Forums section of the Tunisiya corpus because AOC had too few qualifying Maghrebi sentences.
  • The classifiers were Naïve Bayes, SVM, KNN and J48. The baselines were a majority-class classifier (EGY) and a Naïve Bayes model on word uni/bi/trigrams.
  • Accuracy exceeds 76% in training. On unseen data, J48 reaches 66.12% and SVM 64.43%, while n-gram features fall to about 42%.
  • In a side test, a binary MSA-vs-Egyptian n-gram classifier scored near 99% in training and above 95% on unseen data. The binary task is much easier than five-way dialect identification.

Interesting ideas

  • I1. Modelling material that is shared across classes, rather than ignoring it, gave the best features for separating closely related varieties.
  • I2. A tagger trained on the standard variety was deliberately applied to non-standard text, and its imperfect tag distributions still served as useful signals.
  • I3. Evaluating on a completely separate unseen set exposed how badly n-gram features generalize, even though they looked fine in training.
  • I4. Stylistic measures borrowed from authorship attribution (type-token ratio, readability with counts of hard words and syllables) were reused as classification features.
  • I5. Filtering crowd labels by the annotators' own confidence that a sentence was "mostly dialect" produced cleaner class boundaries.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): When converting or parsing across UD dialects/treebanks, we could profile constructions shared by both and focus disambiguation on the residue that actually differs.
  • I2 (tokens): Distributions of tags and function-word classes from the en tagger could flag informal or dialectal English input, without needing the tags to be fully correct.
  • I3 (learn): Every learned component should be scored on a held-out source distinct from its training source, so that lexical memorization is caught before silver data is trusted.
  • I4 (frames): Cheap, explainable register features could inform pragmatic judgements such as formality shifts or irony detection in prag.
  • I5 (learn): Silver examples could be kept only when witness parsers or LLM teachers agree strongly, with mixed cases handled separately rather than forced into a single label.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Lancaster UniversityLancaster · United Kingdom35 works · 1,615 citations🔬British LibraryLondon · United Kingdom2 works · 210 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Mahmoud El-Haj, Paul Rayson and Mariam Aboelezz. 2018. Arabic Dialect Identification in the Context of Bivalency and Code-Switching. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{elhaj2018arabic,
  title = {{Arabic Dialect Identification in the Context of Bivalency and Code-Switching}},
  author = {Mahmoud El-Haj, Paul Rayson and Mariam Aboelezz},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/56ct2dgra5b3},
  url = {https://doi.org/10.63317/56ct2dgra5b3}
}

References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abainia, K., Ouamour, S., and Sayoud, H. (2016). Effective language identification of forum texts based on statistical approaches. Information Processing & Management, 52(4):491 – 512. ↗
  2. Ali, A. M., Bell, P., and Renals, S. (2015). Automatic dialect detection in arabic broadcast speech. Computing Research Repository, abs/1509.06928. ↗
  3. Biadsy, F., Hirschberg, J., and Habash, N. (2009). Spoken arabic dialect identification using phonotactic modeling. In Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, Semitic ’09, pages 53–61, Stroudsburg, PA, USA. Association for Computational Linguistics.
  4. Biber, D. (1988). Variation across speech and writing. Cambridge University Press, Cambridge. find
  5. Blodgett, L. S., Green, L., and O’Connor, B. (2016). Demographic dialectal variation in social media: A case https://github.com/drelhaj/ArabicDialects study of african-american english. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130. Association for Computational Linguistics. ↗
  6. Cavnar, W. B. and Trenkle, J. M. (1994). N-gram-based text categorization. In In Proceedings of SDAIR-94, 3rd Annual Symposium on Document Analysis and Information Retrieval, pages 161–175. find
  7. Dunning, T. (1994). Statistical identification of language. Technical report, New Mexico State University. find
  8. El-Haj, M. and Rayson, P. (2016). Osman – a novel arabic readability metric. In 10th Language Resources and Evaluation Conference (LREC 2016), pages 250–255, Portoroz, Slovenia. find
  9. Elfardy, H., Al-Badrashiny, M., and Diab, M. (2014). Aida: Identifying code switching in informal arabic text. In First Workshop on Computational Approaches to Code Switching. EMNLP 2014, pages 94–101, Doha, Qatar.
  10. Garcia-Barrero, D., Feria, M., and Turell, M. T. (2013). Using function words and punctuation marks in arabic forensic authorship attribution. In Proceedings of the 3rd European Conference of the International Association of Forensic Linguists, pages 42–56, Porto, Portugal. find
  11. Gupta, Kumar, D., Kumar, S., and Ekbal, A. (2015). Machine learning approach for language identification & transliteration. In Proceedings of the Forum for Information Retrieval Evaluation, FIRE ’14, pages 60–64, New York, NY, USA. ACM. find
  12. Habash, N., Rambow, O., Diab, M., and Kanjawi-Faraj, R. (2008). Guidelines for annotation of arabic dialectness. In LREC Workshop on HLT and NLP within the Arabic World, pages 49–53, Marrakesh, Morocco. find
  13. Hofland, K. and Johansson, S. (1982). Word frequencies in British and American English. The Norwegian Computing Centre for the Humanities, Bergen, Norway. find
  14. Holmes, D. I. (1994). Authorship attribution. Computers and the Humanities, 28(2):87–106. find
  15. Laboreiro, G., Bošnjak, M., Sarmento, L., Rodrigues, E. M., and Oliveira, E. (2013). Determining language variant in microblog messages. In Proceedings of the 28th Annual ACM Symposium on Applied Computing, SAC ’13, pages 902–907, New York, NY, USA. ACM. find
  16. Lui, M. and Baldwin, T. (2014). Proceedings of the 5th workshop on language analysis for social media (lasm). pages 17–25. Association for Computational Linguistics. find
  17. Mejdell, G. (2011). Strategic bivalency in written ‘mixed style‘? a reading of ibrahim isa in al-dustur. In Proceedings of the 9th International Arabic Dialectology Association (Aida) Conference, Aida 9th, pages 273–278. find
  18. Ryding, K. (2014). Arabic: A Linguistic Introduction. Cambridge University Press. find
  19. Souter, C., Churcher, G., Hayes, J., Hughes, J., and Johnson, S. (1994). N-gram-based text categorization. In Natural Language Identification using Corpus-based Models, pages 161–175. Hermes Journal of Linguistics. find
  20. Woolard, K. A. and Genovese, E. N. (2007). Strategic bivalency in latin and spanish in early modern spain. Language in Society, 36(4):487–509. ↗
  21. Zaidan, O. F. and Callison-Burch, C. (2014). Arabic dialect identification. Comput. Linguist., 40(1):171–202, March. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/haj-2018-arabic-dialect-identification-context-bivalency/