Elamur Research⌕
☰
home / papers / 2015 / AIDA2: A Hybrid Approach for Token and Sentence Level Dialect Identification in Arabic

AIDA2: A Hybrid Approach for Token and Sentence Level Dialect Identification in Arabic

Paper · 2015 · Mohamed Al-Badrashiny, Heba Elfardy and Mona Diab

Summary

The paper addresses dialect identification in Arabic text where Modern Standard Arabic (MSA) and Egyptian Arabic (EDA) are mixed, sometimes within a single sentence. This is a common pattern in social media, and it hurts tools trained only on MSA. AIDA2, an improved successor of the authors' earlier AIDA system, works at two levels.

  • Token level: each word gets one of six labels from the ShTk code-switching shared task. A CRF sequence labeler makes this decision using signals from MADAMIRA morphology, a dialect-tagged language model, the ModLex modality lexicon, a shallow named-entity module and sentence statistics.
  • Sentence level: two independent classifiers model different aspects of the input. Each uses a feature set chosen heuristically. A decision tree then combines their labels and confidence scores to call the sentence mostly MSA or mostly EDA.

Both levels beat the prior state of the art. Token F-score rises from 86.8% to 90.6%, and sentence accuracy rises from 86.6% to 90.8%. Reliable identification matters because it can serve as a routing step before downstream Arabic tools such as machine translation.

Key points

  • The token tag set is lang1 (MSA), lang2 (EDA), ne, ambig, mixed and other, following the ShTk shared task (Solorio et al., 2014).
  • The language model is a 5-gram model over D3-tokenized text. It is trained on 119K hand-annotated ShTk words plus 8M weblog words (4M MSA, 4M EDA), which are labeled automatically by the dialect of their source.
  • MADAMIRA marks a token as MSA or EDA depending on which analyzer produced its analysis: SAMA gives MSA, CALIMA gives EDA, and no analysis gives unk.
  • At sentence level, Comp-Cl keeps 11 selected features, such as msaPPL, edaPPL, diaPercent and modEDA. Abs-Cl uses only msaProb and edaProb. The decision-tree ensemble ("DT Ensemble") reaches 89.9% in 10-fold cross-validation.
  • In an ablation, removing the modality features lowered Comp-Cl from 87.3% to 87.0%. The authors consider this notable because the feature is so rare.

Interesting ideas

  • I1. The LM training data is labeled for free by source dialect, and each word's label is copied onto all its sub-word pieces. This yields large silver data without manual annotation.
  • I2. Which morphological analyzer succeeds on a word (SAMA vs CALIMA) becomes a feature in its own right. The history of how a word was analyzed is treated as evidence about the word.
  • I3. An "isMixed" flag fires when the prefix, stem and suffix of one word receive different dialect decisions, catching words that blend two varieties.
  • I4. Instead of a majority vote, a decision tree is trained on the labels and confidence scores of two independent classifiers, so it learns when to trust which model.
  • I5. The authors test whether each indicator works in both directions. CALIMA counts flag EDA in 96.5% of EDA sentences, but SAMA counts exceed CALIMA counts in only 23.6% of MSA sentences, so samaWords was dropped.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver training data could inherit labels such as register or genre from document provenance, with the label propagated down to every token of a sentence.
  • I2 (tokens): The en tagger could record which lexicon or rule produced each lemma and expose that origin as an explainable signal for informal or nonstandard words.
  • I3 (tokens): A similar check could flag hybrid forms, such as foreign stems with English affixes, so they are not forced into a single tag.
  • I4 (learn): Witness parsers could be combined by a small learned arbiter over their per-arc confidences, which stays inspectable when it is a shallow tree.
  • I5 (learn): Evaluation could routinely check whether a diagnostic feature discriminates both classes, acting as a negative control against cues that only seem symmetric.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States254 works · 56,747 citations🎓George Washington UniversityWashington · United States51 works · 8,148 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Mohamed Al-Badrashiny, Heba Elfardy and Mona Diab. 2015. AIDA2: A Hybrid Approach for Token and Sentence Level Dialect Identification in Arabic. Proceedings of the Nineteenth Conference on Computational Natural Language Learning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k15-1005

@inproceedings{albadrashiny2015aida2,
  title = {{AIDA2: A Hybrid Approach for Token and Sentence Level Dialect Identification in Arabic}},
  author = {Mohamed Al-Badrashiny, Heba Elfardy and Mona Diab},
  booktitle = {Proceedings of the Nineteenth Conference on Computational Natural Language Learning},
  year = {2015},
  doi = {10.18653/v1/k15-1005},
  url = {https://doi.org/10.18653/v1/k15-1005}
}

References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rania Al-Sabbagh, Jana Diesner, and Roxana Girju. 2013. Using the semantic-syntactic interface for reliable arabic modality annotation. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 410–418, Nagoya, Japan, October. Asian Federation of Natural Language Processing.
  2. Waleed Ammar, Chris Dyer, and Noah A Smith. 2014. Conditional random field autoencoders for unsupervised structured prediction. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3311–3319. Curran Associates, Inc. ↗
  3. Yassine Benajiba, Paolo Rosso, and Jos Miguel Bene-druiz. 2007. Anersys: An arabic named entity recognition system based on maximum entropy. In In Proceedings of CICLing-2007. find
  4. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken arabic dialect identification using phonotactic modeling. In Proceedings of the Workshop on Computational Approaches to Semitic Languages at the meeting of the European Association for Computational Linguistics (EACL), Athens, Greece.
  5. Gokul Chittaranjan, Yogarshi Vyas, Kalika Bali, and Monojit Choudhury, 2014. Proceedings of the First Workshop on Computational Approaches to Code Switching, chapter Word-level Language Identification using CRF: Code-switching Shared Task Report of MSR India System, pages 73–79. Association for Computational Linguistics. find
  6. Ryan Cotterell and Chris Callison-Burch. 2014a. A multi-dialect, multi-genre corpus of informal written arabic. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, may. European Language Resources Association (ELRA).
  7. Ryan Cotterell and Chris Callison-Burch. 2014b. A multi-dialect, multi-genre corpus of informal written arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC).
  8. Heba Elfardy and Mona Diab. 2013. Sentence-Level Dialect Identification in Arabic. In Proceedings of ACL2013, Sofia, Bulgaria, August.
  9. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab, 2014. Proceedings of the First Workshop on Computational Approaches to Code Switching, chapter AIDA: Identifying Code Switching in Informal Arabic Text, pages 94–101. Association for Computational Linguistics.
  10. Jr. Forney, G.D. 1973. The viterbi algorithm. Proceedings of the IEEE, 61(3):268–278, March. ↗
  11. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. Proceedings of the 7th Meeting of the North Amer-icanChapter of the Association for Computational Linguistics/Human Language Technologies Conference (HLT-NAACL06).
  12. Nizar Habash, Ramy Eskander, and AbdelAti Hawwari. 2012. A Morphological Analyzer for Egyptian Arabic. In NAACL-HLT 2012 Workshop on Computational Morphology and Phonology (SIGMORPHON2012), pages 1–9.
  13. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. 2009. The weka data mining software: an update. ACM SIGKDD Explorations Newsletter, 11(1). ↗
  14. Naman Jain and Ahmad Riyaz Bhat, 2014. Proceedings of the First Workshop on Computational Approaches to Code Switching, chapter Language Identification in Code-Switching Scenario, pages 87–93. Association for Computational Linguistics. find
  15. Levi King, Eric Baucom, Timur Gilmanov, Sandra Kübler, Daniel Whyatt, Wolfgang Maier, and Paul Rodrigues. 2014. The iucl+ system: Word-level language identification via extended markov models. In Proceedings of the First Workshop on Computational Approaches to Code-Switching. EMNLP 2014, Conference on Empirical Methods in Natural Language Processing, Doha, Qatar. find
  16. Chu-Cheng Lin, Waleed Ammar, Lori Levin, and Chris Dyer, 2014. Proceedings of the First Workshop on Computational Approaches to Code Switching, chapter The CMU Submission for the Shared Task on Language Identification in Code-Switched Data, pages 80–86. Association for Computational Linguistics. find
  17. Mohamed Maamouri, Dave Graff, Basma Bouziri, Sondos Krouna, Ann Bies, and Seth Kulick. 2010. Ldc standard arabic morphological analyzer (sama) version 3.1. ↗
  18. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M. Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of LREC, Reykjavik, Iceland.
  19. M.A.A. Rashwan, M.A.S.A.A. Al-Badrashiny, M. Attia, S.M. Abdou, and A. Rafea. 2011. A stochastic arabic diacritizer based on a hybrid of factorized and unfactorized textual features. Audio, Speech, and Language Processing, IEEE Transactions on, 19(1):166–175, Jan. ↗
  20. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. 2014. Sentence level dialect identification for machine translation system selection. In Proceedings of the annual meeting of the Association for Computational Linguistics (ACL). find
  21. Fei Sha and Fernando Pereira. 2003. Shallow parsing with conditional random fields. pages 213–220. find
  22. Thamar Solorio, Elizabeth Blair, Suraj Maharjan, Steve Bethard, Mona Diab, Mahmoud Gonheim, Abdelati Hawwari, Fahad AlGhamdi, Julia Hirshberg, Alison Chang, , and Pascale Fung. 2014. Overview for the first shared task on language identification in code-switched data. In In Proceedings of the First Workshop on Computational Approaches to Code-Switching. EMNLP 2014, Conference on Empirical Methods in Natural Language Processing, October, 2014, Doha, Qatar.
  23. Omar F Zaidan and Chris Callison-Burch. 2011. The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In ACL 2011.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/badrashiny-2015-aida2-hybrid-approach-token-sentence/