Elamur Research⌕
☰
home / papers / 2025 / AIN: The Arabic INclusive Large Multimodal Model

AIN: The Arabic INclusive Large Multimodal Model

Summary

Arabic text-only LLMs have improved, but Arabic multimodal models still cover only narrow slices of vision-language understanding. AIN addresses this with a bilingual English-Arabic LMM of 7B parameters, built by full-parameter fine-tuning of Qwen2-VL-7B for a single epoch. The training set has 3.6 million curated multimodal samples, about 35% of which are authentic Arabic. Much of the rest was produced by translation and then checked. The authors tested translation quality with BLEU, ROUGE, METEOR and sentence-embedding similarity, and filtered images for toxicity using LLavaGuard's safety policies together with GPT-4o.

On CAMEL-Bench (8 domains, 38 sub-domains), AIN-7B scores on average 3.4 points above GPT-4o, with the largest advantage on OCR and document understanding. It also beats its base model on ArabicMMLU and on ten English benchmarks, and native speakers in a blind survey preferred its answers.

The work shows that careful data curation can lift a mid-sized open model above much larger closed systems for a language with fewer resources.

Key points

  • AIN-7B is based on Qwen2-VL-7B and was trained with full-parameter fine-tuning for one epoch.
    • Hyper-parameters follow LLaMA-Factory, with flash-attention.
    • Training ran on 8 nodes × 8 A100 80GB GPUs.
  • The data comprises 3.6M samples, roughly 35% genuine Arabic.
    • Measured translation quality: BLEU-4 60.20%, ROUGE-L F1 85.80%, METEOR 86.10%.
    • Toxicity screening judged 95.63% of images safe and 4.37% unsafe; the unsafe part was removed.
  • CAMEL-Bench: AIN-7B gains +3.4 average points over GPT-4o across 38 sub-domains, and is strongest on OCR & document understanding.
  • ArabicMMLU: AIN-7B wins 14 of 19 categories against Qwen2-VL-7B, with a total of 59.36 vs 56.36.
    • On ten English benchmarks it also improves on Qwen2-VL-7B, e.g. MMBench 93.76 vs 81.78.
  • Blind human survey: over 200 native speakers from 17 Arab countries took part.
    • AIN received 76% of the votes, GPT-4o 15% and LLaVA 9%.

Interesting ideas

  • I1. Before using LaBSE or Paraphrase-XLM-R as a translation filter, the authors checked each scorer on 50 known-good and 50 known-bad translations, so the metric was validated before it was trusted. — For Mova (learn): Any agreement score used to accept silver trees or frames should first be run on deliberately corrupted samples, so that a negative control shows it actually separates good from bad.
  • I2. They probed the similarity metric with controlled variants of correct translations (diacritics, punctuation changes, feminine phrasing, purely semantic rewording) and with fully mismatched ones, mapping which differences the score ignores and which it punishes. — For Mova (back): The text → graph → text compressor could be scored by a metric tested the same way, so harmless surface variation is not counted as meaning loss while real content swaps are.
  • I3. In the qualitative comparison, the authors noted when a model ignored the requested answer format (a single letter, one word, or a full sentence), not just when it answered wrongly. — For Mova (answer): School-test and fable-QA readers could get a separate format-compliance check, so that a correct meaning given in the wrong form is diagnosed apart from a reasoning error.
  • I4. Human evaluation hid model identities and asked raters to choose which answer was closest to a given ground truth, rather than give a free preference. — For Mova (answer): Comparing explainable answers against an LLM baseline in such a blind, reference-anchored way would give fairer evidence than open-ended impressions.
  • I5. The training mix combines translated material with a fixed share of native-language data, so the model sees authentic usage and not only translationese. — For Mova (learn): When LLM teacher tiers produce synthetic sentences for training the tagger or parser, a guaranteed share of genuine treebank text could keep the model from drifting toward artificial style.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

11 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2502.00094 · arXiv 2502.00094

@misc{heakl2025ain,
  title = {{AIN: The Arabic INclusive Large Multimodal Model}},
  author = {Heakl, Ahmed and Ghaboura, Sara and Thawkar, Omkar and Khan, Fahad Shahbaz and Cholakkal, Hisham and Anwer, Rao Muhammad and Khan, Salman},
  year = {2025},
  doi = {10.48550/arxiv.2502.00094},
  eprint = {2502.00094},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.00094}
}

References

35 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S Khan, Salman Khan, and Rao M Anwer. Camel-bench: A comprehensive arabic lmm benchmark. arXiv preprint arXiv:2410.18976, 2024. ↗
  2. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. ↗ our card
  3. OpenAI. Gpt-4o model. https://openai.com, 2024. Accessed: 2024. ↗
  4. OpenAI. Gpt-4o-mini model. https://openai.com, 2024. Accessed: 2024-10-14. ↗
  5. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. ↗ our card
  6. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. find
  7. Xiang Yue, Yueqi Song, Akari Asai, Seungone Kim, Jean de Dieu Nyandwi, Simran Khanuja, Anjali Kantharuban, Lintang Sutawika, Sathyanarayanan Ramamoorthy, and Graham Neubig. Pangea: A fully open multilingual multimodal llm for 39 languages. arXiv preprint arXiv:2410.16153, 2024. ↗
  8. Google AI. Gemini: A family of highly capable multimodal models, 2023. find our card
  9. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238, 2023. ↗ our card
  10. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024. ↗ our card
  11. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. ↗ our card
  12. Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, SM Uddin, Shayekh Bin Islam, et al. Maya: An instruction finetuned multilingual multimodal model. arXiv preprint arXiv:2412.07112, 2024. ↗
  13. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Boda Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, et al. Arabicmmlu: Assessing massive multitask language understanding in arabic. arXiv preprint arXiv:2402.12840, 2024. ↗ our card
  14. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision, pages 216–233. Springer, 2025. find our card
  15. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023. ↗ our card
  16. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. find our card
  17. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023. ↗ our card
  18. Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. find
  19. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023. ↗ our card
  20. Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521, 2022. find
  21. Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244, 2022. ↗ our card
  22. Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. Springer, 2016. find our card
  23. Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, jiayi lei, Quanfeng Lu, Peng Gao, Runjian Chen, Peng Xu, Renrui Zhang, Haozhe Zhang, Yali Wang, Yu Qiao, Ping Luo, Kaipeng Zhang, and Wenqi Shao. MMT-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask AGI. In Proceedings of the International Conference on Machine Learning (ICML), 2024. find
  24. OpenAI. Gpt-4 model. https://openai.com, 2024. Accessed: 2024. ↗
  25. Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024. ↗ our card
  26. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1. Minneapolis, Minnesota, 2019. find
  27. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. find
  28. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. Mpnet: Masked and permuted pre-training for language understanding. In Advances in Neural Information Processing Systems, 2020. find our card
  29. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852, 2020. ↗ our card
  30. Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104, 2020. ↗ our card
  31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. find our card
  32. Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005. find our card
  33. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. find
  34. Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, and Patrick Schramowski. Llavaguard: Vlm-based safeguards for vision dataset curation and safety assessment. arXiv preprint arXiv:2406.05113, 2024. ↗
  35. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/heakl-2025-ain-arabic-inclusive-large-multimodal/