Elamur Research⌕
☰
home / papers / 2025 / Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challenges

Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challenges

Summary

This survey maps Arabic Multimodal Machine Learning (MML): systems that combine text, speech, images and video for tasks such as sentiment analysis, emotion recognition, retrieval and speech recognition. The authors argue that Arabic work in this area is now mature enough to review. They also note obstacles specific to Arabic: dialect diversity, complex morphology and few annotated multimodal datasets. They examined 45 studies and sorted them with a new four-axis taxonomy covering datasets, target applications, methodological approaches (from rule-based and classical ML to CNNs, RNNs and Transformers), and core challenges (representation, fusion, translation between modalities, alignment). Their main finding is that the field leans heavily on building datasets and on application studies. Most systems reuse pretrained models built for other languages and combine modalities in simple ways. Harder problems remain largely untouched, including temporal and semantic alignment, robust fusion when inputs are noisy or missing, and efficient Arabic-specific architectures. The survey offers a roadmap for prioritising future Arabic multimodal research.

Key points

  • The review covers 45 papers, grouped by dataset contributions, applications, approaches and challenges.
  • Early datasets include:
    • MMAC (Ashraf et al., 2010): Arabic text images for OCR, either real, synthetic or noisy, with sub-word fragments annotated in two categories: one for word pieces and one for the "naked" versions of those pieces.
    • AVAS (2013): audio-visual speech data with controlled variation in lighting and head pose, intended for AVSR, lip-reading and face recognition.
  • AMMD (Alqarafi et al., 2019) is a sentiment dataset of 830 segments with visual and language modalities. It uses four labels (objective, subjective, positive, negative), and its baselines are SVM-based.
  • Typical pipelines adapt BERT, ResNet or CLIP with little modification and then fuse modalities by concatenation. Fanar Star is an exception, since it was trained from scratch starting with its own data collection.
  • Arabic unimodal LLMs appeared from about 2017, but the first Arabic LMMs only arrived in 2024. Some studies use LLMs to generate synthetic training data when data is scarce.

Interesting ideas

  • I1. MMAC annotates text at several granularities, from full words down to sub-word fragments with and without diacritic dots, so that systems can learn from pieces smaller than a word. — For Mova (tokens): Clitics, contractions and hyphenated fragments could get their own annotated units in the tokenizer's test data, so that errors can be traced to a specific sub-word segment.
  • I2. AVAS varies lighting and pose on purpose, which turns robustness into a property that can be measured rather than assumed. — For Mova (learn): Evaluation sets could be built with controlled perturbations such as casing, punctuation loss and typos, and each run compared against the clean version as a negative control.
  • I3. The survey opens with the observation that sarcasm often cannot be recovered from the words alone and depends on extra context. — For Mova (frames): The irony detector in prag could explicitly flag cases where the text by itself is insufficient evidence, instead of making a confident guess, which keeps the decision explainable.
  • I4. The paper names fusion under missing or noisy inputs, and the choice between early, late and hybrid fusion, as an open core problem. — For Mova (learn): Combining witness parsers into silver data is also a fusion problem, so the system could test late voting against hybrid schemes and specify behaviour when one witness is absent or unreliable.
  • I5. The four-axis taxonomy (data × application × approach × challenge) is used as a matrix that exposes empty cells, which correspond to research gaps. — For Mova (learn): A similar coverage grid of pipeline stage × phenomenon × evaluation type would show which parts of Mova have no tests or training data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2508.12227 · arXiv 2508.12227

@misc{haouhat2025arabic,
  title = {{Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challenges}},
  author = {Haouhat, Abdelhamid and Bellaouar, Slimane and Nehar, Attia and Cherroun, Hadda and Abdelali, Ahmed},
  year = {2025},
  doi = {10.48550/arxiv.2508.12227},
  eprint = {2508.12227},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.12227}
}

References

71 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samah Abbas, Dimah Alahmadi, and Hassanin Al-Barhamtoshy. Establishing a multimodal dataset for arabic sign language (arsl) production. Journal of King Saud University - Computer and Information Sciences, 36(8):102165, 2024. find
  2. Ashraf AbdelRaouf, Colin A Higgins, Tony Pridmore, and Mahmoud Khalil. Building a multi-modal arabic corpus (MMAC). International Journal on Document Analysis and Recognition (IJDAR), 13(4):285–302, December 2010. find
  3. Ftoon Abu Shaqra, Rehab Duwairi, and Mahmoud Al-Ayyoub. The audio-visual arabic dataset for natural emotions. In 2019 7th International Conference on Future Internet of Things and Cloud (FiCloud), pages 324–329, 2019. find
  4. Ftoon Abu Shaqra, Rehab Duwairi, and Mahmoud Al-Ayyoub. A multi-modal deep learning system for arabic emotion recognition. Int. J. Speech Technol., 26(1):123–139, June 2022. find
  5. Sadam Al-Azani and El-Sayed M. El-Alfy. Enhanced video analytics for sentiment analysis based on fusing textual, auditory and visual information. IEEE Access, 8:136843–136857, 2020. find
  6. Noora Al Roken and Gerassimos Barlas. Multimodal arabic emotion recognition using deep learning. Speech Communication, 155:103005, 2023. find
  7. Sagheer Alaa. Multimodal arabic speech recognition for human-robot interaction applications. Applied Mathematics & Information Sciences: Vol. 09: Iss. 6, Article 15., 2015. find
  8. Sohaila Alalem, Mohamed Saad Zaghloul, and Osama Badawy. A novel deep learning multi-modal sentiment analysis model for english and egyptian arabic dialects using audio and text. In 2023 24th International Arab Conference on Information Technology (ACIT), pages 1–5, 2023. find
  9. Rasha M. Albalawi, Amani T. Jamal, Alaa O. Khadidos, and Areej M. Alhothali. Multimodal arabic rumors detection. IEEE Access, 11:9716–9730, 2023. find
  10. Raed Alharbi. Mf-saudi: A multimodal framework for bridging the gap between audio and textual data for saudi dialect detection. Journal of King Saud University - Computer and Information Sciences, 36(6):102084, 2024. find
  11. Abdulrahman S. Alqarafi, Ahsan Adeel, Mandar Gogate, Kia Dashitpour, Amir Hussain, and Tariq Durrani. Toward’s arabic multi-modal sentiment analysis. In Qilian Liang, Jiasong Mu, Min Jia, Wei Wang, Xuhong Feng, and Baoju Zhang, editors, Communications, Signal Processing, and Systems, pages 2378–2386, Singapore, 2019. Springer Singapore. find
  12. Norah Alrashoudi, Hend Al-Khalifa, and Yousef Alotaibi. Improving mispronunciation detection and diagnosis for non-native learners of the arabic language. Discover Computing, 28(1):1, Jan 2025. find
  13. Norah A. Alsharif, Yakoub Bazi, and Mohamad M. Al Rahhal. Learning to align arabic and english text to remote sensing images using transformers. In 2022 IEEE Mediterranean and Middle-East Geoscience and Remote Sensing Symposium (M2GARSS), pages 9–12, 2022. find
  14. Fakhraddin Alwajih, Gagan Bhatia, and Muhammad Abdul-Mageed. Dallah: A dialect-aware multimodal large language model for Arabic. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 320–336, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  15. Fakhraddin Alwajih, El Moatez Billah Nagoudi, Gagan Bhatia, Abdelrahman Mohamed, and Muhammad Abdul-Mageed. Peacock: A family of Arabic multimodal large language models and benchmarks. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12753–12776, Bangkok, Thailand, August 2024. Association Manuscript submitted to ACM for Computational Linguistics. find
  16. Samar Antar, Alaa Sagheer, Saleh Aly, and Mohamed F. Tolba. AVAS: Speech database for multimodal recognition applications. In 13th International Conference on Hybrid Intelligent Systems (HIS 2013), pages 123–128, 2013. find
  17. Ehsaneddin Asgari, Yassine El Kheir, and Mohammad Ali Sadraei Javaheri. Morphbpe: A morpho-aware tokenizer bridging linguistic complexity for efficient llm training across morphologies, 2025. find
  18. Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell., 41(2):423–443, February 2019. find
  19. Arnab Barua, Mobyen Uddin Ahmed, and Shahina Begum. A systematic literature review on multimodal machine learning: Applications, challenges, gaps and future directions. IEEE Access, 11:14804–14831, 2023. find
  20. Mohamed Lazhar Bellagha and Mounir Zrigui. Using the MGB-2 challenge data for creating a new multimodal dataset for speaker role recognition in Arabic TV Broadcasts. Procedia Computer Science, 192:59–68, 2021. Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 25th International Conference KES2021. find
  21. Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih, and Muhammad Abdul-Mageed. Qalam: A multimodal LLM for Arabic optical character and handwriting recognition. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 210–224, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  22. Maximilian Bisani and Hermann Ney. Joint-sequence models for grapheme-to-phoneme conversion. Speech Communication, 50(5):434–451, 2008. find
  23. Dmitry Bogdanov, Nicolas Wack, Emilia Gómez, Sankalp Gulati, Perfecto Herrera, Oscar Mayor, Gerard Roma, Justin Salamon, José Zapata, and Xavier Serra. Essentia: an open-source library for sound and music analysis. In Proceedings of the 21st ACM International Conference on Multimedia, MM ’13, page 855–858, New York, NY, USA, 2013. Association for Computing Machinery. find
  24. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606, 2016. ↗
  25. Amal Dandashi, Jihad Aljaam, and Sebti Foufou. Audio-visual video classification system design: For arabic news domain. In 2016 International Conference on Computational Science and Computational Intelligence (CSCI), pages 745–751, 2016. find
  26. Amal Dandashi and Jihad Mohamad Alja’am. Video classification methods: Multimodal techniques. In Jihad Mohamad Alja’am, Abdulmotaleb El Saddik, and Abdul Hamid Sadka, editors, Recent Trends in Computer Applications, pages 33–51, Cham, 2018. Springer International Publishing. find
  27. Samar Elbedwehy and T. Medhat. Improved arabic image captioning model using feature concatenation with pre-trained word embedding. Neural Computing and Applications, 35(26):19051–19067, Sep 2023. find
  28. Dan Wang Emran Al-Buraihy. Enhancing cross-lingual image description: A multimodal approach for semantic relevance and stylistic alignment. Computers, Materials & Continua, 79(3):3913–3938, 2024. find
  29. Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S. Khan, Salman Khan, and Rao M. Anwer. Camel-bench: A comprehensive arabic lmm benchmark, 2024. find
  30. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Manuscript submitted to ACM Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. find
  31. Jonatas Grosman. Fine-tuned XLSR-53 large model for speech recognition in Arabic. https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-arabic, 2021. ↗
  32. Abdelhamid Haouhat, Slimane Bellaouar, Attia Nehar, and Hadda Cherroun. Towards arabic multimodal dataset for sentiment analysis. In 2023 Fourth International Conference on Intelligent Data Science Technologies and Applications (IDSTA), pages 126–133, 2023. find our card
  33. Abdelhamid Haouhat, Slimane Bellaouar, Attia Nehar, and Hadda Cherroun. Amd’saer: Arabic multimodal dataset for sentiment analysis and emotion recognition. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 2024. Submitted for publication. find
  34. Abdelhamid Haouhat, Hadda Cherroun, Slimane Bellaouar, and Attia Nehar. MODOS at ArAIEval shared task: Multimodal propagandistic memes classification using weighted SAM, CLIP and ArabianGPT. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 483–488, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  35. Maram Hasanain, Md. Arid Hasan, Fatema Ahmad, Reem Suwaileh, Md. Rafiul Biswas, Wajdi Zaghouani, and Firoj Alam. ArAIEval shared task: Propagandistic techniques detection in unimodal and multimodal Arabic content. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 456–466, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  36. Ahmed Heakl, Sara Ghaboura, Omkar Thawkar, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, and Salman Khan. Ain: The arabic inclusive large multimodal model. arXiv preprint arXiv:2502.00094, 2025. ↗ our card
  37. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. Manuscript submitted to ACM find
  38. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460, 2021. find our card
  39. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. Fasttext.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651, 2016. ↗ our card
  40. Abdel Ghani Karkar, Jihad Mohamad Alja’am, and Arif Mahmood. Illustrate it! an arabic multimedia text-to-picture m-learning system. IEEE Access, 5:12777–12787, 2017. find
  41. S. P. Khandait, R. C. Thool, and P. D. Khandait. Automatic facial feature extraction and expression recognition based on neural network, 2012. find
  42. Taehyeong Kim, Hyeonseop Song, and Byoung-Tak Zhang. Cross-modal alignment learning of vision-language conceptual systems, 2022. find
  43. Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. Foundations & trends in multimodal machine learning: Principles, challenges, and open questions. ACM Comput. Surv., 56(10), June 2024. find
  44. Pengfei Liu, Kun Li, and Helen Meng. Group gated fusion on attention-based bidirectional alignment for multimodal emotion recognition, 2022. find
  45. David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, Nov 2004. find
  46. Hamzah Luqman. Arabsign: A multi-modality dataset and benchmark for continuous arabic sign language recognition. In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), page 1–8. IEEE Press, 2023. find
  47. Imad Eddine Marouf, Enzo Tartaglione, Stephane Lathuiliere, and Joost van de Weijer. No images, no problem: Retaining knowledge in continual vqa with questions-only memory, 2025. find
  48. Sunakshi Mehra, Virender Ranga, Ritu Agarwal, and Seba Susan. Speaker independent recognition of low-resourced multilingual arabic spoken words through hybrid fusion. Multimedia Tools and Applications, March 2024. find
  49. Mohamed Moselhy, Abdallah Bastawy, Osama Yousef, Mohamed Khalil, Mohamed Ashraf, Ibrahim Elsayed, and Yasser Hifny. Multimodal arabic retrieval and recommendation system. In 2023 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pages 1–7, 2023. find
  50. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. Crosslingual generalization through multitask finetuning. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15991–16111, Toronto, Canada, July 2023. Association for Computational Linguistics. find our card
  51. Michele Mukeshimana, Abraham Niyongere, and Jérémie Ndikumagenge. Facial emotion recognition feature extraction: A survey. In Seyyed Abed Hosseini, editor, Emotion Recognition, chapter 4. IntechOpen, Rijeka, 2023. find
  52. Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, and Simon Kornblith. Human alignment of neural network representations, 2025. find
  53. Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. Multimodal deep learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, page 689–696, Madison, WI, USA, 2011. Omnipress. find
  54. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Manuscript submitted to ACM Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. find our card
  55. Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37:98–125, 2017. find
  56. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. find our card
  57. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision, 2022. find our card
  58. Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre-training. 2018. find
  59. Mohamad M. Al Rahhal, Yakoub Bazi, Norah A. Alsharif, Laila Bashmal, Naif Alajlan, and Farid Melgani. Multilanguage transformer for improved text to remote sensing image retrieval. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15:9115–9126, 2022. find
  60. Moutaz Saleh and Jihad Mohamad Alja’am. Towards adaptive multimedia system for assisting children with arabic learning difficulties. In 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), pages 794–799, 2019. find
  61. Uzair Shah, Md. Rafiul Biswas, Marco Agus, Mowafa Househ, and Wajdi Zaghouani. MemeMind at ArAIEval shared task: Generative augmentation and feature fusion for multimodal propaganda detection in Arabic memes through advanced language and vision models. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 467–472, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  62. Zien Sheikh Ali, Watheq Mansour, Tamer Elsayed, and Abdulaziz Al-Ali. AraFacts: The first large Arabic dataset of naturally occurring claims. In Nizar Habash, Houda Bouamor, Hazem Hajj, Walid Magdy, Wajdi Zaghouani, Fethi Bougares, Nadi Tomeh, Ibrahim Abu Farha, and Samia Touileb, editors, Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 231–236, Kyiv, Ukraine (Virtual), April 2021. Association for Computational Linguistics. find
  63. Zhiqiang Shi and Ruchit Agrawal. A comprehensive survey of contemporary arabic sentiment analysis: Methods, challenges, and future directions, 2025. find
  64. Linxuan Song, Wenxuan Tu, Sihang Zhou, and En Zhu. Gann: Graph alignment neural network for semi-supervised learning. Pattern Recognition, 154:110484, 2024. find
  65. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus’ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, and Chaoyi Ruan. Fanar: An arabic-centric multimodal generative ai platform, 2025. find our card
  66. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc. find
  67. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, volume 1, pages I–I, 2001. find
  68. Yeshan Wang and Ilia Markov. CLTL at ArAIEval shared task: Multimodal propagandistic memes classification using transformer models. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 501–506, Bangkok, Thailand, August 2024. Association for Computational Linguistics. find
  69. Fatima Zohra Youcef and Fatiha Barigou. Arabic language investigation in the context of unimodal and multimodal sentiment analysis. In 2021 22nd International Arab Conference on Information Technology (ACIT), pages 1–7, 2021. find
  70. Muhy Eddin Za’ter and Bashar Talafha. Bench-marking and improving arabic automatic image captioning through the use of multi-task learning paradigm. ArXiv, abs/2202.05474, 2022. ↗
  71. Mohamed Zaytoon, Nagwa El-Makky, and Marwan Torki. AlexUNLP-MZ at ArAIEval shared task: Contrastive learning, LLM features extraction and multi-objective optimization for Arabic multi-modal meme propaganda detection. In Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, and Khalil Mrini, editors, Proceedings of The Second Arabic Natural Language Processing Conference, pages 512–517, Bangkok, Thailand, August 2024. Association for Computational Linguistics. Manuscript submitted to ACM find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/haouhat-2025-arabic-multimodal-machine-learning-datasets/