Elamur Research⌕
☰
home / papers / 2024 / DiaSet: An Annotated Dataset of Arabic Conversations

DiaSet: An Annotated Dataset of Arabic Conversations

Paper · 2024 · Abraham Israeli, Aviv Naaman, Guy Maduel et al.

Summary

DiaSet tackles the shortage of NLP resources for spoken Palestinian Arabic (PAL), a South Levantine dialect used in Palestine, Israel and Jordan. Most Arabic corpora are written, and only a small fraction come from audio. Dialect work has also concentrated on Egyptian and Moroccan. The authors built a corpus of transcribed conversations from two sources. The first is natural podcast dialogues between YouTube influencers and their guests. The second is interviews the team recorded with native speakers from many localities, which ensures that urban, rural, Bedouin and Druze varieties of the dialect all appear in the data. Transcription followed one consistent spelling convention, which matters because dialectal Arabic has no standard written form. Part of the material was hand-labelled for sentiment analysis (SA) and named entity recognition (NER), and baselines were trained by fine-tuning existing Arabic BERT models. The authors describe these as the first SA and NER models built specifically for PAL. The dataset is released publicly and gives a starting point for dialect-aware processing of conversational speech.

Key points

  • The corpus contains 644.8K tokens and 23.2K annotated instances, and each annotated unit carries both SA and NER labels.
  • Transcribed audio totals about 83 hours of YouTube podcasts and 15 hours of purpose-recorded interviews.
  • Every conversation has two speakers, an interviewer and an interviewee, and all of them are native PAL speakers.
  • The paper's motivating statistic is that only 16.8% of Arabic NLP corpora come from audio sources.
  • The paper gives examples of PAL-specific lexicon, for instance issa for "now" and hāy for "this". It also notes Hebrew loanwords among Arab citizens of Israel and Egyptian influence in Gaza.

Interesting ideas

  • I1. The corpus mixes naturally occurring talk (podcasts) with elicited interviews of speakers chosen from varied regions, so both natural register and coverage of sub-varieties are secured.
  • I2. A single fixed transcription convention was enforced for a language variety that has no standard orthography, which keeps downstream token statistics consistent.
  • I3. The same text units are annotated for two different tasks, so one resource supports several layers and lets the layers be compared on identical input.
  • I4. A small table of dialect-marker words, each contrasted with its standard and neighbouring-dialect equivalents, works as a compact diagnostic of variety-specific lexicon.
  • I5. Interview participants signed an explicit consent form before recording, which builds ethical provenance into the data pipeline.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation sets could combine found text with deliberately commissioned texts that target under-covered constructions or genres, so that gaps in coverage show up.
  • I2 (tokens): When Mova ingests informal English (speech transcripts, chat), a written normalization policy for contractions, fillers and spelling variants would keep tokenizer and lemma behaviour stable.
  • I3 (frames): Labelling one passage set for entities, sentiment or speech acts, and roles together would let Mova check whether the frames layer and the entity layer agree on the same sentences.
  • I4 (world): A similar contrast table of regional or register-specific English terms could feed the lexicon and ontology, so variant words map to the same concept in the world model.
  • I5 (learn): Any human-elicited data for Mova (e.g., test answers or written stories) could carry a recorded consent and provenance tag, so training sets remain auditable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Abraham Israeli, Aviv Naaman, Guy Maduel, Rawaa Makhoul, Dana Qaraeen, Amir Ejmail, Dina Lisnanskey, Julian Jubran, Shai Fine and Kfir Bar. 2024. DiaSet: An Annotated Dataset of Arabic Conversations. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{israeli2024diaset,
  title = {{DiaSet: An Annotated Dataset of Arabic Conversations}},
  author = {Abraham Israeli, Aviv Naaman, Guy Maduel, Rawaa Makhoul, Dana Qaraeen, Amir Ejmail, Dina Lisnanskey, Julian Jubran, Shai Fine and Kfir Bar},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  doi = {10.63317/4a4hbsyte2n7},
  url = {https://doi.org/10.63317/4a4hbsyte2n7}
}

References

57 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16.
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. Arbert & marbert: Deep bidirectional transformers for arabic. arXiv preprint arXiv:2101.01785. ↗
  3. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics.
  4. Mahmoud Al-Ayyoub, Abed Allah Khamaiseh, Yaser Jararweh, and Mohammed N Al-Kabi. 2019. A comprehensive survey of arabic sentiment analysis. Information processing & management, 56(2):320–342. ↗
  5. Sadam Al-Azani and El-Sayed M El-Alfy. 2019. Audio-textual arabic dialect identification for opinion mining videos. In 2019 IEEE symposium series on computational intelligence (SSCI), pages 2470–2475. IEEE. find
  6. Anhar Al-Mutlaq. 2017. Tebyan: Interactive spelling correction application for quranic verse. In 2017 9th IEEE-GCC Conference and Exhibition (GC-CCE), pages 1–9. IEEE. find
  7. Brahim Ait Ben Ali, Soukaina Mihi, Ismail El Bazi, and Nabil Laachfoubi. 2020. A recent survey of arabic named entity recognition on social media. Rev. d’Intelligence Artif., 34(2):125–135. ↗
  8. Zaid Alyafeai, Maraim Masoud, Mustafa Ghaleb, and Maged S. Al-shaibani. 2021. Masader: Metadata sourcing for arabic text and speech data resources.
  9. Muhammad Amara. 2017. Arabic in Israel: Language, identity and conflict. Routledge. find
  10. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗
  11. Naaima Boudad, Rdouan Faizi, Rachid Oulad Haj Thami, and Raddouane Chiheb. 2018. Sentiment analysis in arabic: A review of the literature. Ain Shams Engineering Journal, 9(4):2479–2490. ↗
  12. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. ↗
  13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. ↗
  14. William M Cotter. 2013. Dialect contact and change in gaza city. Unpublished MA thesis. Colchester, UK: University of Essex. ↗
  15. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R El-Beltagy, Wassim El-Hajj, et al. 2021. A panoramic survey of natural language processing in the arab world. Communications of the ACM, 64(4):72–81. ↗
  16. Louise Deleger, Qi Li, Todd Lingren, Megan Kaiser, Katalin Molnar, Laura Stoutenborough, Michal Kouril, Keith Marsolo, Imre Solti, et al. 2012. Building gold standard corpora for medical natural language processing tasks. In AMIA Annual Symposium Proceedings, volume 2012, page 144. American Medical Informatics Association. find
  17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  18. Shahd Dibas, Christian Khairallah, Nizar Habash, Omar Fayez Sadi, Tariq Sairafy, Karmel Sarabta, and Abrar Ardah. 2022. Maknuune: A large open palestinian arabic lexicon. arXiv preprint arXiv:2210.12985. ↗
  19. Paul Ekman et al. 1999. Basic emotions. Handbook of cognition and emotion, 98(45-60):16. find
  20. Mohamed Eldesouki, Naassih Gopee, Ahmed Ali, and Kareem Darwish. 2019. Farspeech: Arabic natural language processing for live arabic speech. In INTERSPEECH, pages 2372–2373. find
  21. Ashraf Elnagar, Sane Yagi, Ali Bou Nassif, Ismail Shahin, and Said A. Salloum. 2021. Sentiment analysis in dialectal arabic: A systematic review. In Advanced Machine Learning Technologies and Applications, pages 407–417, Cham. Springer International Publishing. find
  22. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online arabic sentiment analyser. In Proceedings of the fourth arabic natural language processing workshop, pages 192–198.
  23. Ibrahim Abu Farha and Walid Magdy. 2022. The effect of arabic dialect familiarity on data annotation. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 399–408.
  24. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in arabic. In Proceedings of the sixth Arabic natural language processing workshop, pages 296–305.
  25. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2021a. Arabic natural language processing: An overview. Journal of King Saud University-Computer and Information Sciences, 33(5):497–507.
  26. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2021b. Arabic natural language processing: An overview. Journal of King Saud University - Computer and Information Sciences, 33(5):497–507.
  27. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9.
  28. Nizar Y Habash. 2022. Introduction to Arabic natural language processing. Springer Nature.
  29. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021a. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics.
  30. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021b. The interplay of variant, size, and task type in arabic pre-trained language models. arXiv preprint arXiv:2103.06678. ↗
  31. Abraham Israeli, Aviv Naaman, Yotam Nahum, Razan Assi, Shai Fine, and Kfir Bar. 2022. Love me, love me not: Human-directed sentiment analysis in arabic. In Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-located with ICNLSP 2022, pages 22–30.
  32. Abraham Israeli, Yotam Nahum, Shai Fine, and Kfir Bar. 2021. The idc system for sentiment classification and sarcasm detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 370–375.
  33. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a corpus for palestinian Arabic: a preliminary study. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 18–27, Doha, Qatar. Association for Computational Linguistics.
  34. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested arabic named entity corpus and recognition using bert. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3626–3636.
  35. Serena Jeblee, Weston Feely, Houda Bouamor, Alon Lavie, Nizar Habash, and Kemal Oflazer. 2014. Domain and dialect adaptation for machine translation into egyptian arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 196–206.
  36. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. ↗
  37. Wuwei Lan, Yang Chen, Wei Xu, and Alan Ritter. 2020. Gigabert: Zero-shot transfer learning from english to arabic. In Proceedings of The 2020 Conference on Empirical Methods on Natural Language Processing (EMNLP). find
  38. Bing Liu et al. 2010. Sentiment analysis and subjectivity. Handbook of natural language processing, 2(2010):627–666. find
  39. Diego Mollá, Menno Van Zaanen, and Daniel Smith. 2006. Named entity recognition for question answering. In Proceedings of the Australasian language technology workshop 2006, pages 51–58. find
  40. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. arXiv preprint arXiv:2003.07082. ↗
  41. Xiaoye Qu, Yingjie Gu, Qingrong Xia, Zechang Li, Zhefeng Wang, and Baoxing Huai. 2023. A survey on arabic named entity recognition: Past, recent advances, and future trends. arXiv preprint arXiv:2302.03512. ↗
  42. Erik F Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050.
  43. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, et al. 2023. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv preprint arXiv:2308.16149. ↗
  44. Khaled Shaalan. 2014. A survey of arabic named entity recognition and classification. Computational Linguistics, 40(2):469–510. ↗
  45. Ali Shazal, Aiza Usman, and Nizar Habash. 2020. A unified model for arabizi detection and transliteration using sequence-to-sequence models. In Proceedings of the fifth arabic natural language processing workshop, pages 167–177.
  46. Abdulhadi Shoufan and Sumaya Alameri. 2015. Natural language processing for dialectical arabic: A survey. In Proceedings of the second workshop on Arabic natural language processing, pages 36–48.
  47. Abu Bakr Soliman, Kareem Eissa, and Samhaa R El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  48. Jindian Su and Hong Yu. 2023. Unified named entity recognition as multi-label sequence generation. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE. find
  49. Kapil Tajane, Saransh Dave, Pankaj Jahagirdar, Abhijeet Ghadge, and Akash Musale. 2018. Ai based chat-bot using azure cognitive services. In 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), pages 1–4. IEEE. find
  50. Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020-2022. Label Studio: Data labeling software. Open source software available from https://github.com/heartexlabs/label-studio. ↗
  51. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682. ↗
  52. WorldData. 2023. Arabic speaking countries. https://www.worlddata.info/languages/arabic.php#google_vignette. Accessed: 2023-08-30. 10. ↗
  53. Language Resource References Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2022. Hierarchical aggregation of dialectal data for arabic dialect identification. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4586–4596.
  54. Alex Brandsen, Suzan Verberne, Milco Wansleeben, and Karsten Lambers. 2020. Creating a dataset for named entity recognition in the archaeology domain. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4573–4577. ↗
  55. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a corpus for palestinian Arabic: a preliminary study. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 18–27, Doha, Qatar. Association for Computational Linguistics.
  56. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: an annotated corpus for the palestinian arabic dialect. Language Resources and Evaluation, 51:745–775. ↗
  57. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/israeli-2024-diaset-annotated-dataset-arabic-conversations/