Elamur Research⌕
☰
home / papers / 2026 / AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia

AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia

Paper · 2026 · Sultan Alrowili, Younes Samih, Abed Alhakim Freihat et al.

Summary

Arabic vision–language models have few resources for testing encyclopedic, fact-based knowledge. Existing Arabic VQA sets mostly cover culture or dialects, are small and hand-curated, or are not public. The authors built an automatic pipeline over the Arabic Wikipedia dump. It uses MediaWiki template tags to split articles into sections and link each section to its single image, caption and context text. It then reconstructs full image URLs from MD5 hashes of filenames and fetches license data from Wikidata. Finally, GPT-OSS-120B turns each image-plus-context pair into a four-option multiple-choice question, and malformed outputs are filtered out.

The result is AraVQA: about 50.7k questions over roughly 47k images, covering more than 20 general-knowledge subjects. A test subset was checked by human annotators. Fine-tuning small VLMs on the training split raised accuracy by up to 20 points on Pearl and on the AraVQA test set, and the tuned models beat 70B+ models. The pipeline runs on one GPU and could be reused for other Wikipedia languages.

Key points

  • AraVQA has 50,757 questions over 46,958 images, all generated automatically, with train and test splits. Earlier Arabic sets such as Henna, CAMEL-Bench, JEEM and Pearl provide test splits only.
  • Question generation used GPT-OSS-120B with vLLM on a single NVIDIA H100.
    • Quality control found over 94% of generated questions valid.
    • The benchmark subset was verified by six human annotators.
  • Preprocessing rules:
    • Images were downscaled so their longest side is at most 224 px; 768 and 1024 px gave only marginal gains.
    • Sections with several images were dropped.
    • Sections over 1,024 tokens were dropped.
    • Entries without a valid image URL were dropped.
  • Granite-3.3-2B (Vision) and Phi-4-Multimodal were fine-tuned on 45,969 examples. They gained up to 20 pp and outperformed InternVL2-Llama3-76B, Qwen2-VL-72B-Instruct and Llama-4-Scout-17B-16E-Instruct.
  • Resources are released at github.com/mbzuai-nlp/AraVQA. Known gaps: little OCR or logic-reasoning coverage, and larger VLMs were not fine-tuned because they ran out of memory on one GPU.

Interesting ideas

  • I1. Pair each image only with the text section it belongs to, and throw away sections with several images, so the context that grounds a question is never ambiguous.
  • I2. After generation, reject outputs that break the template (missing options, placeholder text), then shuffle the four answers and reassign the gold label so the correct letter is evenly distributed.
  • I3. Generate data synthetically at scale, then measure a validity rate on a sample and keep a separately human-verified slice as the benchmark.
  • I4. Work with cheap low-resolution copies but keep pointers to the originals (URLs, licenses recovered from Wikidata P275), so the data stays light yet traceable.
  • I5. A small model fine-tuned on narrowly targeted, well-filtered data can beat far larger general models on that task.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): when loading facts from Wikipedia or Wikidata, keep only claims tied unambiguously to one entity's section, so the world graph does not merge context belonging to neighbouring entities.
  • I2 (answer): auto-built multiple-choice tests over fables and school texts should get the same shuffle, so a reader cannot score by guessing a favoured position, and that baseline can serve as a sanity check.
  • I3 (learn): questions or silver labels produced by the LLM teacher tier can be audited the same way, with a sampled validity percentage plus a small hand-checked gold set kept apart from training.
  • I4 (world): every compiled knowledge item could store a source pointer and license field, keeping answers explainable back to their origin without storing full documents.
  • I5 (learn): this supports training compact task-specific components on curated silver data rather than relying on a large general model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBMArmonk · United States139 works · 70,691 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 2,908 worksVision & language · 657 worksQuestion answering · 441 works

Cite

Cite as Sultan Alrowili, Younes Samih, Abed Alhakim Freihat and Mathan Kumar Eswaran. 2026. AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.91

@inproceedings{alrowili2026aravqa,
  title = {{AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia}},
  author = {Sultan Alrowili, Younes Samih, Abed Alhakim Freihat and Mathan Kumar Eswaran},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.91},
  url = {https://doi.org/10.18653/v1/2026.acl-long.91}
}

References

23 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024c. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198.
  2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, and 8 others. 2022. Flamingo: a visual language model for few-shot learning. Preprint, arXiv:2204.14198. ↗
  3. Yazeed Alnumay, Alexandre Barbet, Anna Bialas, William Darling, Shaan Desai, Joan Devassy, Kyle Duffy, Stephanie Howe, Olivia Lasche, Justin Lee, Anirudh Shrinivason, and Jennifer Tracey. 2025. Command r7b arabic: A small, enterprise focused, multilingual, and culturally aware arabic llm. Preprint, arXiv:2503.14603. Fakhraddin Alwajih, Samar Mohamed Magdy, Abdellah El Mekki, Omer Nacar, Youssef Nafea, Safaa Taher Abdelfadil, Abdulfattah Mohammed Yahya, Hamzah Luqman, Nada Almarwani, Samah Aloufi, Baraah Qawasmeh, Houdaifa Atou, Serry Sibaee, Hamzah A. Alsayadi, Walid Al-Dhabyani, Maged S. Al-shaibani, Aya El aatar, Nour Qandos, Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, and 1 others. 2024b. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821. ↗
  4. Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Husain Salem Abdulla Alharthi, Ines Riahi, Ab-duljalil Radman, Jorma Laaksonen, Fahad Shahbaz Khan, Salman Khan, and Rao Muhammad Anwer. 2025. CAMEL-bench: A comprehensive Arabic LMM benchmark. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 1970–1980, Albuquerque, New Mexico. Association for Computational Linguistics. ↗
  5. Momchil Hardalov, Todor Mihaylov, Dimitrina Zlatkova, Yoan Dinkov, Ivan Koychev, and Preslav Nakov. 2020. EXAMS: A multi-subject high school examinations dataset for cross-lingual and multilingual question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5427–5444, Online. Association for Computational Linguistics. find
  6. Ahmed Heakl, Sara Ghaboura, Omkar Thawkar, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, and Salman Khan. 2025. Ain: The arabic inclusive large multimodal model. Preprint, arXiv:2502.00094. ↗
  7. Karima Kadaoui, Hanin Atwany, Hamdan Al-Ali, Abdelrahman Mohamed, Ali Mekky, Sergei Tilga, Natalia Fedorova, Ekaterina Artemova, Hanan Aldarmaki, and Yova Kementchedjhieva. 2025. Jeem: Vision-language understanding in four arabic dialects. Preprint, arXiv:2503.21910. ↗
  8. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin. 2024. ArabicMMLU: Assessing massive multitask language understanding in Arabic. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5622–5640, Bangkok, Thailand. Association for Computational Linguistics. ↗
  9. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  10. J Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33 1:159–74. ↗
  11. Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2023a. Mimic-it: Multi-modal in-context instruction tuning. Preprint, arXiv:2306.05425. ↗
  12. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Preprint, arXiv:2301.12597. ↗
  13. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. Visual instruction tuning. Preprint, arXiv:2304.08485. ↗
  14. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b. Visual instruction tuning. Preprint, arXiv:2304.08485. ↗
  15. Meta. 2025. Llama-4-scout-17b-16e-instruct. Machine learning model. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. Preprint, arXiv:2503.01743. ↗
  16. Mistral AI. 2025. Mistral-Small-3.2-24B-Instruct-2506. https://huggingface.co/mistralai/ Mistral-Small-3.2-24B-Instruct-2506. ↗
  17. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, and 13 others. 2023. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. Preprint, arXiv:2308.16149. ↗
  18. Granite Vision Team, Leonid Karlinsky, Assaf Arbelle, Abraham Daniels, Ahmed Nassar, Amit Al-fassi, Bo Wu, Eli Schwartz, Dhiraj Joshi, Jovana Kondic, Nimrod Shabtay, Pengyuan Li, Roei Herzig, Shafiq Abedin, Shaked Perek, Sivan Harary, Udi Barzelay, Adi Raz Goldfarb, Aude Oliva, and 44 others. 2025. Granite vision: a lightweight, open-source multimodal model for enterprise intelligence. Preprint, arXiv:2502.09927. ↗
  19. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. ↗
  20. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  21. Zhiyang Xu, Ying Shen, and Lifu Huang. 2023. Multiinstruct: Improving multi-modal zero-shot learning via instruction tuning. Preprint, arXiv:2212.10773. ↗
  22. Meta AI. 2024. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783. Llama 3.2-11B-Vision-Instruct model, updated September 25, 2024. ↗
  23. Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, and 57 others. 2025. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alrowili-2026-aravqa-building-new-arabic-factoid/