Elamur Research⌕
☰
home / papers / 2025 / PhantomTroupe at ImageEval 2025 Shared Task: Multimodal Arabic Image Captioning through Translation-Based Fine-Tuning of LLM Models

PhantomTroupe at ImageEval 2025 Shared Task: Multimodal Arabic Image Captioning through Translation-Based Fine-Tuning of LLM Models

Paper · 2025 · Muhammad Abu Horaira, Farhan Amin, Sakibul Hasan et al.

Summary

The authors tackle Arabic image captioning for culturally loaded pictures, mostly tied to Palestinian heritage, in Subtask 2 of the ImageEval 2025 shared task. The task dataset is small and manually captioned. Instead of training a model to write Arabic directly, the team moved the supervision into English. They used unsloth/Qwen3-14B to translate the Arabic reference captions, fine-tuned a vision-language model on image–English pairs, and then translated the generated captions back into Arabic. The final system is Qwen2.5-VL-7B-Instruct, quantized to 4 bits and adapted with LoRA through PEFT. Images were converted to grayscale and resized before training. They compared it with Llama 3.2 Vision (11B) and Pixtral (12B), and Qwen scored best on all three metrics: BLEU-1, cosine similarity and an LLM judge. The system placed third in the competition. The work suggests that a mid-sized, instruction-tuned multimodal model, using a strong language as a pivot, can be competitive when target-language data is scarce. The remaining weaknesses are dialect handling and dependence on translation quality.

Key points

  • Dataset: 3,071 manually annotated images, split into 2,717 train, 75 validation and 279 blind test. Evaluation ran on CodaLab.
  • Final model: unsloth/Qwen2.5-VL-7B-Instruct-bnb-4bit with NF4 quantization (BitsAndBytes).
    • LoRA was applied to both the vision and language parts, with rank 64, alpha 64 and dropout 0.
    • Training used TRL SFTTrainer for 3 epochs, batch size 32, learning rate 5e-5, a cosine schedule and AdamW 8-bit.
    • Initial runs used rank 16, alpha 32, dropout 0.05 and batch size 4.
  • Scores (BLEU-1 / cosine / LLM judge):
    • Qwen2.5-VL 7B: 22.6 / 57.48 / 31.43
    • Llama 3.2 11B: 18.9 / 49.32 / 26.75
    • Pixtral 12B: 15.4 / 42.19 / 22.10
  • Preprocessing: all images were converted to grayscale and resized to 224×224. Few images in the set were RGB, and the authors prioritised structure and texture over colour.
  • Errors: the system mixes dialects, defaults to Modern Standard Arabic and sometimes drops culturally important terms. Translation errors can propagate into the final captions.

Interesting ideas

  • I1. Supervise the model in a well-resourced pivot language, then translate its output back into the target language, so pretraining strength carries over to a scarce target language.
  • I2. The translator was chosen by checking how much meaning survives a round trip through it, not by its one-way fluency.
  • I3. The same instruction–response template was used in fine-tuning and at inference, so train and test prompts never diverge.
  • I4. Captions were scored with three complementary measures (n-gram overlap, embedding similarity and an LLM judge), because each catches failures the others miss.
  • I5. The error analysis specifically tracked whether culturally specific named terms survived the pipeline, not only overall caption quality.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Translation through the meta-world could be benchmarked against this simple text-pivot baseline to show what the meaning-graph route actually adds.
  • I2 (learn): The text → graph → text compressor could use round-trip meaning loss as a cheap signal for ranking or rejecting silver parses.
  • I3 (learn): Prompts for the LLM teacher tiers could be frozen into one shared template, so that teacher labels and later queries are comparable.
  • I4 (back): Generated text could be judged by surface overlap, semantic similarity and a judge together, with disagreements flagged for inspection.
  • I5 (world): Checking that every entity in the world of the text reappears after generation or translation would give an explainable fidelity check.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh28 works · 29 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksVision & language · 811 works

Cite

Cite as Muhammad Abu Horaira, Farhan Amin, Sakibul Hasan, Md. Tanvir Ahammed Shawon and Muhammad Ibrahim Khan. 2025. PhantomTroupe at ImageEval 2025 Shared Task: Multimodal Arabic Image Captioning through Translation-Based Fine-Tuning of LLM Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.57

@inproceedings{horaira2025phantomtroupe,
  title = {{PhantomTroupe at ImageEval 2025 Shared Task: Multimodal Arabic Image Captioning through Translation-Based Fine-Tuning of LLM Models}},
  author = {Muhammad Abu Horaira, Farhan Amin, Sakibul Hasan, Md. Tanvir Ahammed Shawon and Muhammad Ibrahim Khan},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.57},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.57}
}

References

11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Imad Afyouni, Imtinan Azhar, and Ashraf Elnagar. 2021. Aracap: A hybrid deep learning architecture for arabic image captioning. Procedia Computer Science, 189:171–178. find
  2. Hend Al-Khalifa, Mustafa Jarrar, and Wajdi Zaghouani. 2021. Challenges in arabic image captioning: A review. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(4):1– 25. find
  3. Batool Mohammed Alazzam. 2022. Arabic image captioning using resnet50. Journal of Al-Qadisiyah for Computer Science and Mathematics, 14(1):81–88. Ahlam Bashiti, Alaa Aljabari, Hadi Hamoud, Md. find
  4. Rafiul Biswas, Bilal Shalash, Mustafa Jarrar, Fadi Zaraket, George Mikros, Ehsaneddin Asgari, and Wajdi Zaghouani. 2025. Imageeval 2025: The first arabic image captioning shared task. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  5. Samar Elbedwehy and Tamer Medhat. 2023. Improved arabic image captioning model using feature concatenation with pre-trained word embedding. Neural Computing and Applications, 35(25):18575–18592. find
  6. Hani D. Hejazi and Khaled Shaalan. 2021. Deep learning for arabic image captioning: A comparative study of main factors and preprocessing recommendations. International Journal of Advanced Computer Science and Applications, 12(11):37–44. find
  7. Haneen Siraj Ibrahim, Narjis Mezaal Shatia, and AbdulRahman A. Alsewari. 2024. A transfer learning approach for arabic image captions. Mustansiriyah Journal of Science, 35(1):70–79. find
  8. Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA. IEEE.
  9. Abdelrahman M. Sayed, Mohamed K. Elhadad, Gouda I. Salama, and Aiman M. Mousa. 2024. Improving arabic image captioning with vision-language models. In Proceedings of the 10th International Conference on Electrical Engineering and Informatics (ICEEI), Bandung, Indonesia. IEEE. find
  10. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France. PMLR.
  11. Muhy Eddin Za’ter and Bashar Talafha. 2022. Benchmarking and improving arabic automatic image captioning through the use of multi-task learning paradigm. arXiv preprint arXiv:2202.05474. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/horaira-2025-phantomtroupe-imageeval-2025-shared-task/