Elamur Research⌕
☰
home / papers / 2023 / OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media

OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media

Summary

Arabic speakers on social media mostly write in regional dialects, not Modern Standard Arabic (MSA). MT systems built for MSA handle this poorly. Earlier dialectal parallel corpora are small and tied to narrow domains such as the Bible or travel phrases. They also ignore online slang and idioms, and they rarely document how translators were chosen or how translations were checked.

The authors built OSN-MDAD to fill this gap. English tweets were translated into four urban dialects (Gulf, Yemeni, Iraqi, Levantine/Shami) following a written guideline framework. The framework favours contextual, localized translation over word-for-word renderings and is meant to transfer to other language–dialect pairs.

To show the data is usable, they fine-tuned mBART separately for each dialect and compared it with seq2seq-with-attention models trained from scratch. The fine-tuned Transformers reached roughly 30–42 points on a combined BLEU/ROUGE F-score, while the baselines stayed at or below 13.1. The paper positions the resource as a benchmark for informal English→dialectal Arabic translation.

Key points

  • Dialects covered: Gulf, Yemeni, Iraqi and Levantine/Shami. The same English tweets were rendered into every dialect, giving parallel versions across varieties.
  • Gaps in prior work: The authors list nine weaknesses of earlier corpora (Zbib et al.; Bouamor et al., including the BTEC-based city-level corpus). These include:
    • small size, sometimes about 2K sentences per dialect;
    • unscreened translators;
    • domain dependence;
    • no handling of idioms, code-borrowing or online cultural expressions;
    • missing or vague guidelines.
  • Models and setup: One mBART model was fine-tuned per dialect with default settings, using a 90/10 train/test split. Baselines were seq2seq-with-attention models trained from scratch.
  • Results (combined BLEU/ROUGE F-score):
Dialectseq2seq baselineFine-tuned mBARTGain
Iraqi1.741.7about 40 points
Gulf13.133.8about 21 points
Levantine/Shami1.734.6about 33 points
Yemeni11.3about 30about 19 points
  • Limitations and future work: Only short texts were tested. Future plans include adding more dialects, handling long sentences, and moving to evaluation that is more linguistically informed than BLEU.

Interesting ideas

  • I1. Translating one identical source set into every target variety yields a fully aligned multi-variety corpus, so differences in output can be traced to the variety rather than to the content. — For Mova (tree): Parsing the same sentences in different registers or UD dialect conversions would isolate parser errors caused by style, separating them from errors caused by sentence content.
  • I2. Annotators are screened against explicit criteria, such as native command of the target variety and active familiarity with social-media slang, and all criteria are written down. — For Mova (learn): LLM teacher tiers and witness parsers could carry a similar documented qualification profile per text genre, so silver data comes only from sources proven competent on that genre.
  • I3. Idioms and culturally loaded phrases are translated by their meaning in context ("localization") instead of word by word. — For Mova (back): Translating through the meta-world should map an idiom to its frame-level meaning first and then regenerate a natural target expression, rather than aligning surface tokens.
  • I4. The paper argues that single-domain corpora (Bible, travel phrases) cause out-of-vocabulary and out-of-domain failures, so it samples from open, informal social-media text. — For Mova (learn): Evaluation sets for the readers should include open-domain informal text, as a check that skills learned on fables and school texts actually generalize.
  • I5. The authors concede that an aggregate BLEU score hides where translations go wrong, and they propose linguistically grounded error analysis instead. — For Mova (answer): Students' results could be broken down by linguistic phenomenon (negation, coreference, idiom) instead of reporting one accuracy figure, which makes failures explainable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Machine translation

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.12137 · arXiv 2309.12137

@misc{alzamzami2023osn,
  title = {{OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media}},
  author = {Alzamzami, Fatimah and Saddik, Abdulmotaleb El},
  year = {2023},
  doi = {10.48550/arxiv.2309.12137},
  eprint = {2309.12137},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.12137}
}

References

37 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ibrahim Abu Farha and Walid Magdy. From arabic sentiment analysis to sarcasm detection: The arsarcasm dataset. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39, 2020. find our card
  2. Alexandra Balahur and Marco Turchi. Multilingual sentiment analysis using machine translation? In Proceedings of the 3rd workshop in computational approaches to subjectivity and sentiment analysis, pages 52–60, 2012. find
  3. Kia Dashtipour, Soujanya Poria, Amir Hussain, Erik Cambria, Ahmad YA Hawalah, Alexander Gelbukh, and Qiang Zhou. Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cognitive computation, 8(4):757–771, 2016. find
  4. Keita Fujihira and Noriko Horibe. Multilingual sentiment analysis for web text based on word to word translation. In 2020 9th International Congress on Advanced Applied Informatics (IIAI-AAI), pages 74–79, 2020. find
  5. Huu-anh Tran, Yuhang Guo, Ping Jian, Shumin Shi, and Heyan Huang. Improving parallel corpus quality for chinese-vietnamese statistical machine translation. Journal of Beijing Institute of Technology, 27(1), 2018. find
  6. Mehreen Alam and Sibt Ul Hussain. Roman-urdu-parl: Roman-urdu and urdu parallel corpus for urdu language understanding. Transactions on Asian and Low-Resource Language Information Processing, 21(1):1–20, 2022. find
  7. Najet Hadj Mohamed, Malak Rassem, Lifeng Han, and Goran Nenadic. Alphamwe-arabic: Arabic edition of multilingual parallel corpora with multiword expression annotations. 2023. find
  8. Marcus Tomalin, Bill Byrne, Shauna Concannon, Danielle Saunders, and Stefanie Ullmann. The practical ethics of bias reduction in machine translation: Why domain adaptation is better than data debiasing. Ethics and Information Technology, pages 1–15, 2021. find
  9. Donia Gamal, Marco Alfonse, Salud Marîa Jiménez-Zafra, and Mostafa Aref. Survey of arabic machine translation, methodologies, progress, and challenges. In 2022 2nd International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pages 378–383. IEEE, 2022. find
  10. Baligh Babaali and Mohammed Salem. Survey of the arabic machine translation corpora. In Modelling and Implementation of Complex Systems: Proceedings of the 7th International Symposium, MISC 2022, Mostaganem, Algeria, October 30-31, 2022, pages 205–219. Springer, 2022. find
  11. Saméh Kchaou, Rahma Boujelbane, and Lamia Hadrich. Hybrid pipeline for building arabic tunisian dialect-standard arabic neural machine translation model from scratch. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(3):1–21, 2023. find
  12. Toshiyuki Takezawa, Genichiro Kikui, Masahide Mizushima, and Eiichiro Sumita. Multilingual spoken language corpus development for communication research. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 3, September 2007: Special Issue on Invited Papers from ISCSLP 2006, pages 303–324, 2007. find
  13. Jezia Zakraoui, Moutaz Saleh, Somaya Al-Maadeed, and Jihad Mohamed Alja’am. Arabic machine translation: A survey with challenges and future directions. IEEE Access, 9:161445–161468, 2021. find
  14. Houda Bouamor, Nizar Habash, and Kemal Oflazer. A multidialectal parallel corpus of arabic. In LREC, pages 1240–1245, 2014. find our card
  15. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar Zaidan, and Chris Callison-Burch. Machine translation of arabic dialects. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 49–59, 2012. find our card
  16. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, et al. The madar arabic dialect corpus and lexicon. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), 2018. find
  17. Abida Emna, Saméh Kchaou, and Rahma Boujelban. Neural machine translation of low resource languages: Application to transcriptions of tunisian dialect. In Intelligent Systems and Pattern Recognition: Second International Conference, ISPR 2022, Hammamet, Tunisia, March 24–26, 2022, Revised Selected Papers, pages 234–247. Springer, 2022. find
  18. Ali Bou Nassif, Abdollah Masoud Darya, and Ashraf Elnagar. Empirical evaluation of shallow and deep learning classifiers for arabic sentiment analysis. Transactions on Asian and Low-Resource Language Information Processing, 21(1):1–25, 2021. find our card
  19. Hassan Sajjad, Ahmed Abdelali, Nadir Durrani, and Fahim Dalvi. Arabench: Benchmarking dialectal arabic-english machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5094–5107, 2020. find
  20. Wael Salloum and Nizar Habash. Dialectal to standard arabic paraphrasing to improve arabic-english statistical machine translation. In Proceedings of the first workshop on algorithms and resources for modelling of dialects and language varieties, pages 10–21, 2011. find
  21. Hassan Sajjad, Nadir Durrani, Francisco Guzman, Preslav Nakov, Ahmed Abdelali, Stephan Vogel, Wael Salloum, Ahmed El Kholy, and Nizar Habash. Egyptian arabic to english statistical machine translation system for nist openmt’2015. arXiv preprint arXiv:1606.05759, 2016. ↗
  22. Wael Salloum and Nizar Habash. Dialectal arabic to english machine translation: Pivoting through modern standard arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 348–358, 2013. find
  23. Marta R Costa-Jussa, Marcos Zampieri, and Santanu Pal. A neural approach to language variety translation. arXiv preprint arXiv:1807.00651, 2018. ↗
  24. Laith H Baniata, Sangwoo Kang, and Isaac KE Ampomah. A reverse positional encoding multi-head attention-based neural machine translation model for arabic dialects. Mathematics, 10(19):3666, 2022. find
  25. Amel Slim, Ahlem Melouah, Usef Faghihi, and Khouloud Sahib. Improving neural machine translation for low resource algerian dialect by transductive transfer learning strategy. Arabian Journal for Science and Engineering, 47(8):10411–10418, 2022. find
  26. Laith H Baniata, Seyoung Park, and Seong-Bae Park. A neural machine translation model for arabic dialects that utilizes multitask learning (mtl). Computational intelligence and neuroscience, 2018, 2018. find
  27. Pamela Shapiro and Kevin Duh. Comparing pipelined and integrated approaches to dialectal arabic neural machine translation. In Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 214–222, 2019. find
  28. Frank Fujita, Ed Diener, and Ed Sandvik. Gender differences in negative affect and well-being: the case for emotional intensity. Journal of personality and social psychology, 61(3):427, 1991. find
  29. Anna Wierzbicka. Emotions across languages and cultures: Diversity and universals. Cambridge University Press, 1999. find
  30. Rana Abaalkhail, Fatimah Alzamzami, Samah Aloufi, Rajwa Alharthi, and Abdulmotaleb El Saddik. Affectional ontology and multimedia dataset for sentiment analysis. In International Conference on Smart Multimedia, pages 15–28. Springer, 2018. find
  31. Majid Mohammed Ali Mabkhut Musaad and Adbulrahman Ali Al Towity. Translation evaluation of three machine translation systems, with special references to idiomatic expressions. AF Cd˜ ¤ T§wrt˜ wl`˜ Tl› TyžAsž³ , (29):678–708, 2023. find
  32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017. find
  33. Fatimah Alzamzami and Abdulmotaleb El Saddik. Monitoring cyber sentihate social behavior during covid-19 pandemic in north america. IEEE Access, 2021. find
  34. Haopeng Wang, M Shamim Hossain, Abdulmotaleb El Saddik, et al. Deep learning (dl)-enabled system for emotional big data. IEEE Access, 9:116073–116082, 2021. find
  35. Fatimah Alzamzami, Mohamad Hoda, and Abdulmotaleb El Saddik. Light gradient boosting machine for general sentiment classification on short texts: A comparative evaluation. IEEE Access, 2020. find
  36. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742, 2020. find
  37. Shumin Shi, Xing Wu, Rihai Su, and Heyan Huang. Low-resource neural machine translation: Methods and trends. ACM Transactions on Asian and Low-Resource Language Information Processing, 21(5):1–22, 2022. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alzamzami-2023-osn-mdad-machine-translation-dataset/