Elamur Research⌕
☰
home / papers / 2020 / Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System

Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System

Paper · 2020 · Rawan Tahssin, Youssef Kishk and Marwan Torki

Summary

The paper describes the CodeLyoko team's entry to the NADI 2020 Shared Task, Subtask-1. The task is to guess which of 21 Arab countries a tweet's dialect comes from. Two things make it hard: the classes are heavily imbalanced, and neighbouring dialects share much vocabulary. To rebalance the data without any external labelled corpus, the authors built RTCES (Reverse Translation Corpus Extension System). It sends each cleaned tweet through Google's NMT into English and back into Arabic, then keeps only the words that differ between the original and the round-tripped version. These leftovers tend to be the dialect-specific words, because the standard-Arabic material survives translation largely unchanged. The leftovers become new training sentences for the smaller classes.

The authors tested four classifiers on this extended corpus:

  • TF-IDF features with logistic regression
  • averaged fastText vectors with logistic regression
  • fastText vectors with a BiLSTM
  • fine-tuned AraBERT

AraBERT did best, reaching a 20.34% macro F1 on test and placing 7th of 18. The work shows a cheap, label-free way to rebalance data for fine-grained dialect classification.

Key points

  • Task and data: NADI 2020 Subtask-1 covers country-level identification across 21 dialects, using only the official dataset. The extended training corpus after RTCES holds 32,417 sentences.

  • Selective augmentation: RTCES sentences were added for every class except the three largest (Egypt, Iraq, Saudi Arabia).

  • Dev-set macro F1:

    ModelMacro F1
    TF-IDF + LR17.34
    fastText + LR14.24
    fastText + BiLSTM18.33
    AraBERT, fine-tuned after Farasa segmentation20.73

  • Final test results (AraBERT): 20.34 F1, 36.26 accuracy, 27.83 precision and 20.56 recall. These are close to the dev numbers, which the authors read as a sign of no overfitting.

  • Hierarchical variant: The authors also tried a two-level scheme. It first predicts one of five regional groups (Maghreb, Egypt/Sudan, Gulf, Levant, Others), then the country. With TF-IDF + LR, the second level scored roughly at the fastText + LR level.

Interesting ideas

  • I1. Translate a text into a pivot language and back, then diff it against the original: whatever the translation alters or drops marks nonstandard, region-specific wording rather than the shared standard core.
  • I2. Generate synthetic examples only for minority classes and leave the dominant classes untouched, so augmentation actively rebalances the label distribution instead of merely inflating it.
  • I3. Split a large confusable label set into coarse families first and pick the fine label within the chosen family, which makes the confusions easier to inspect.
  • I4. Keep foreign-language words and emojis as features during cleanup, since in noisy social text they can carry class signal.
  • I5. Compare a simple TF-IDF baseline against neural models, and use the dev–test gap as a sanity check on overfitting.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A round-trip diff could flag dialectal, slang or idiomatic English tokens that the tagger and lemmatizer may mishandle, so they get explicit lexicon checks instead of silent guesses.
  • I2 (learn): When building silver treebanks, extra paraphrased or perturbed sentences could be produced only for rare constructions or relations, evening out what the parsers are trained on.
  • I3 (frames): Speech-act or VerbNet-class assignment could first pick a broad family and then a specific class, with each level logged as a separate, explainable decision.
  • I4 (tokens): The tokenizer could preserve emojis and code-switched words as typed tokens instead of discarding them, so later stages such as irony detection can use them.
  • I5 (learn): Every new parser or reader could be reported next to a trivial baseline and on two held-out splits, so a gain is only trusted if it beats the baseline and holds on both.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Rawan Tahssin, Youssef Kishk and Marwan Torki. 2020. Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System. Proceedings of the Fifth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tahssin2020identifying,
  title = {{Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System}},
  author = {Rawan Tahssin, Youssef Kishk and Marwan Torki},
  booktitle = {Proceedings of the Fifth Arabic Natural Language Processing Workshop},
  year = {2020},
  doi = {10.13140/rg.2.2.31541.91363},
  url = {https://doi.org/10.13140/rg.2.2.31541.91363}
}

References

15 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and H. Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In HLT-NAACL Demos.
  2. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop (WANLP 2020), Barcelona, Spain. ↗
  3. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. ↗
  4. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy, August. Association for Computational Linguistics.
  5. François Chollet. 2015. keras. https://github.com/fchollet/keras. ↗
  6. Gael de Francony, Victor Guichard, Praveen Joshi, Haithem Afli, and Abdessalam Bouchekif. 2019. Hierarchical deep learning for Arabic dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 249–253, Florence, Italy, August. Association for Computational Linguistics.
  7. Youssef Fares, Zeyad El-Zanaty, Kareem Abdel-Salam, Muhammed Ezzeldin, Aliaa Mohamed, Karim El-Awaad, and Marwan Torki. 2019. Arabic dialect identification with deep learning and hybrid frequency based features. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 224–228, Florence, Italy, August. Association for Computational Linguistics.
  8. A. Graves and J. Schmidhuber. 2005. Framewise phoneme classification with bidirectional lstm networks. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 4, pages 2047– 2052 vol. 4. find
  9. Mai Ibrahim, Marwan Torki, and Nagwa El-Makky. 2018. Imbalanced toxic comments classification using data augmentation and deep learning. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 875–878, Dec. find
  10. Mai Ibrahim, Marwan Torki, and Nagwa El-Makky. 2020. Alexu-backtranslation-tl at semeval-2020 task [12]: Improving offensive language detection using data augmentation and transfer learning. In Proceedings of the International Workshop on Semantic Evaluation (SemEval). ↗
  11. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA).
  12. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in python. J. Mach. Learn. Res., 12(null):2825–2830, November. ↗
  13. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta, May. ELRA. find
  14. Amira Shoukry and Ahmed Rafea. 2012. Preprocessing egyptian dialect tweets for sentiment mining. 11.
  15. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tahssin-2020-identifying-nuanced-dialect-arabic-tweets/