Elamur Research⌕
☰
home / papers / 2018 / Image Classification for Arabic: Assessing the Accuracy of Direct English to Arabic Translations

Image Classification for Arabic: Assessing the Accuracy of Direct English to Arabic Translations

Paper · 2018 · Abdulkareem Alsudais

Summary

Most image classification work assumes English labels, so Arabic-speaking users of vision tools such as object readers for the visually impaired are left out. This paper asks a simple question: if ImageNet's English labels (WordNet synsets) are run through an off-the-shelf online translation service, how often is the Arabic label correct? The author randomly sampled 10,000 ImageNet images. 2,887 of them were still reachable online. Each image's English label was machine-translated into Arabic, and a human judge rated whether the Arabic output correctly named the depicted object. About two thirds of the translations (65.6%) were judged accurate, and single-word labels did best. The study offers this figure as a baseline for Arabic image classification built on translation. Because a correct judgment transfers to every image sharing that synset, the work also releases a large Arabic-labelled image set. It further sorts the translation failures into types, which point toward better pipelines, for example ones that add context or handle fine-grained categories differently.

Key points

  • The sample began at 10,000 random ImageNet images and shrank to 2,887 after images no longer accessible online were dropped.
  • An online English→Arabic translation API produced the labels, and a single human judge evaluated them. Some outputs were rated "neutral" rather than forced into right or wrong.
  • 65.6% of the Arabic labels were accurate. One-word labels scored highest, and fine-grained categories such as specific dog or bird breeds raised the error rate.
  • 1,895 correctly labelled images cover 1,643 unique synsets. Extending each verified synset to all of its ImageNet images yields 1,910,935 Arabic-labelled images.
  • The paper describes four notable output types:
    • the wrong sense of an ambiguous word;
    • a whole Arabic definition instead of a single word ("pretzel" came out as a dry, salty cake);
    • a correct but rare word that the judge could only confirm with a dictionary (e.g. "earthwork", "teasel");
    • the English word simply written in Arabic script (e.g. "Tiramisu").

Interesting ideas

  • I1. Verify a label once per synset and then propagate it to every instance of that synset, turning a few thousand human checks into almost two million labelled items.
  • I2. Instead of a single accuracy number, sort translation outputs into a taxonomy: wrong sense, definition in place of a word, correct but rare, and transliteration.
  • I3. Many errors came from the translator picking the wrong sense, so the author suggests supplying context about the concept before translating it.
  • I4. For fine-grained categories, translate the broad hypernyms ("bird", "animal") first and then work down to their subcategories.
  • I5. Give the human judge a "neutral" option for cases they cannot confirm, instead of forcing a correct/incorrect call.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could confirm silver annotations at the level of lexical entries or verb classes, then spread each confirmed decision across the whole silver corpus to save teacher budget.
  • I2 (back): Mova's graph→text and translation-through-meta-world outputs could be tagged with the same failure types, making regressions explainable rather than just counted.
  • I3 (world): when mapping concepts across languages, Mova should pass the WordNet synset id or gloss, never a bare lemma, so each meaning stays tied to its ontology node.
  • I4 (world): Mova's ontology could ground or label general nodes first and let specific breeds or species fall back to their parent when no reliable name exists.
  • I5 (learn): Mova's evaluations could add an explicit abstain class, so that uncertain gold items do not quietly inflate or deflate scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Prince Sattam Bin Abdulaziz UniversityAl Kharj · Saudi Arabia17 works · 489 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSemitic languages · 2,908 works

Cite

Cite as Abdulkareem Alsudais. 2018. Image Classification for Arabic: Assessing the Accuracy of Direct English to Arabic Translations. arXiv:1807.05206 · IEEE Access 2019.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/access.2019.2926924 · arXiv 1807.05206

@misc{alsudais2018image,
  title = {{Image Classification for Arabic: Assessing the Accuracy of Direct English to Arabic Translations}},
  author = {Abdulkareem Alsudais},
  year = {2018},
  doi = {10.1109/access.2019.2926924},
  eprint = {1807.05206},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/access.2019.2926924}
}

References

42 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Z. Akata, F. Perronnin, Z. Harchaoui, and C. Schmid, “Good practice in large-scale learning for image classification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 3, pp. 507–520, 2014. ↗
  2. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255. find
  3. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-fei, “ImageNet large scale visual recognition challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, 2015. ↗
  4. F. S. Al-anzi and D. Abuzeina, “Toward an enhanced Arabic text classification using cosine similarity and Latent Semantic Indexing,” J. King Saud Univ. - Comput. Inf. Sci., vol. 29, no. 2, pp. 189–195, 2017. ↗
  5. A. Al-Thubaity, M. Alhoshan, and I. Hazzaa, “Using word N-Grams as features in Arabic text classification,” in Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, R. Lee, Ed. Cham: Springer International Publishing, 2015, pp. 35–43. ↗
  6. T. Kanan and E. A. Fox, “Automated Arabic text classification with p-stemmer, machine learning, and a tailored news article taxonomy,” J. Assoc. Inf. Sci. Technol., vol. 67, no. 11, pp. 2667– 2683, 2016. find
  7. D. Zhang, H. Xu, Z. Su, and Y. Xu, “Chinese comments sentiment classification based on word2vec and SVMperf,” Expert Syst. Appl., vol. 42, no. 4, pp. 1857–1863, 2015. ↗
  8. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. ↗
  9. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition.,” arXiv Prepr., 2015.
  10. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, 2015, pp. 91– 99. ↗
  11. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
  12. J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, and K. Murphy, “Speed/accuracy trade-offs for modern convolutional object detectors,” in CVPR, 2017, pp. 7310–7319. find
  13. Y. Xian, B. Schiele, and Z. Akata, “Zero-shot learning- The good, the bad and the Ugly,” in Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, 2016. find
  14. J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in arXiv preprint:1612.08242, 2016. ↗
  15. J. Dai, Y. Li, K. He, and J. Sun, “R-FCN: object detection via region-based fully convolutional networks,” arXiv: 1605.06409, 2016. ↗
  16. A. Gaidon, Q. Wang, Y. Cabon, and E. Vig, “Virtual worlds as proxy for multi-object tracking analysis,” in CVPR, 2016, pp. 4340–4349. ↗
  17. D. Held, S. Thrun, and S. Savarese, “Learning to track at 100 FPS with deep regression networks,” arXiv:1604.01802, 2016. ↗
  18. A. Karpathy and L. Fei-fei, “Deep visual-demantic alignments for generating image descriptions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 664–676, 2017. find
  19. K. Xu, J. L. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in Proceedings of the 32nd International Conference on Machine Learning, 2015. ↗
  20. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “VQA: visual question answering,” in International Conference on Computer Vision (ICCV), 2015, pp. 2425–2433. ↗
  21. Z. Yang, X. He, J. Gao, L. Deng, and A. Smola, “Stacked attention networks for image question answering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 21–29.
  22. G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis,” Med. Image Anal., vol. 42, pp. 60–88, 2017. ↗
  23. K. Shaalan and H. Raza, “NERA: Named entity recognition for Arabic,” J. Assoc. Inf. Sci. Technol., vol. 60, no. 8, pp. 1652–1663, 2009. ↗
  24. Ma. Oudah and K. Shaalan, “NERA 2.0: Improving coverage and performance of rule-based named entity recognition for Arabic,” Nat. Lang. Eng., vol. 23, no. 3, pp. 441–472, 2016. ↗
  25. M. Rushdi-saleh, M. T. Martín-valdivia, L. A. Ureña-lópez, and J. M. Perea-ortega, “OCA: Opinion corpus for Arabic,” J. Assoc. Inf. Sci. Technol., vol. 62, no. 10, pp. 2045–2054, 2011. ↗
  26. M. Al-smadi, B. Talafha, M. Al-Ayyoub, and Y. Jararweh, “Using long short ‑ term memory deep neural networks for aspect ‑ based sentiment analysis of Arabic reviews,” Int. J. Mach. Learn. Cybern., 2018. ↗
  27. A. Almuhareb, W. Alsanie, and A. Al-thubaity, “Arabic word segmentation with long short-term memory neural networks and word embedding,” IEEE Access, vol. 7, pp. 12879–12887, 2019. ↗
  28. M. Nicosia, S. Filice, A. Barron-Cedeno, I. Saleh, H. Mubarak, W. Gao, P. Nakov, G. Da San Martino, A. Moschitti, K. Darwish, L. Marquez, S. Joty, and W. Magdy, “QCRI: Answer selection for community question answering – experiments for Arabic and English,” in Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), 2015, pp. 203–209.
  29. A. Azmi and N. Alshenaifi, “Lemaza: An Arabic why-question answering system,” Nat. Lang. Eng., 2017. ↗
  30. S. L. Marie-sainte, N. Alalyani, S. Alotaibi, S. Ghouzali, and I. Abunadi, “Arabic natural language processing and machine learning-based systems,” IEEE Access, vol. 7, pp. 7011–7020, 2019. ↗
  31. S. A. Salloum, M. Al-emran, and K. Shaalan, “A Survey of lexical functional grammar in the Arabic context,” Int. J. Com. Net. Tech, vol. 4, no. 3, 2016. find
  32. Y. Samih, M. Attia, M. Eldesouki, H. Mubarak, A. Abdelali, L. Kallmeyer, and K. Darwish, “A Neural architecture for dialectal Arabic segmentation,” in Proceedings of the Third Arabic Natural Language Processing Workshop, 2017, pp. 46–54.
  33. N. Zalmout, A. Erdmann, and N. Habash, “Noise-robust morphological disambiguation for dialectal Arabic,” in Proceedings of NAACL-HLT, 2018, pp. 953–964.
  34. I. Zeroual and A. Lakhouaja, “Arabic corpus linguistics: Major progress, but still a long way to go,” in Intelligent Natural Language Processing: Trends and Applications, K. Shaalan, A. E. Hassanien, and F. Tolba, Eds. Cham: Springer International Publishing, 2018, pp. 613–636. ↗
  35. S. Roller and S. Schulte, “A Multimodal LDA Model Integrating Textual, Cognitive and Visual Modalities,” in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013, no. October, pp. 1146–1157. find
  36. H. A. Al-muzaini, T. N. Al-yahya, and H. Benhidour, “Automatic Arabic Image Captioning using RNN-LSTM-Based Language Model and CNN,” Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 6, pp. 67–73, 2018. find
  37. V. Jindal, “Generating image captions in Arabic using root-word based recurrent neural networks and deep neural networks,” in Proceedings of NAACL-HLT 2018: Student Research Workshop, 2018, pp. 144–151.
  38. K. Papineni, S. Roukos, T. Ward, and W. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in In Proceedings of the 40th annual meeting on association for compu- tational linguistics, 2002, no. July, pp. 311–318.
  39. R. Al-shalabi, G. Kanaan, H. Al-Sarhan, A. Drabsh, and I. Al-Husban, “Evaluating machine translations from Arabic into English and vice versa,” Int. Res. J. Electron. Comput. Eng., vol. 3, no. 2, 2017. ↗
  40. W. Black, S. Elkateb, H. Rodriguez, M. Alkhalifa, P. Vossen, A. Pease, and C. Fellbaum, “Introducing the Arabic WordNet Project,” in Proceedings of the third international WordNet conference., 2006, pp. 295–299. find
  41. Y. Regragui, L. Abouenour, F. Krieche, K. Bouzoubaa, and P. Rosso, “Arabic WordNet: New content and new applications Arabic WordNet,” in Proceedings of the Eighth Global WordNet Conference, 2016, no. January, pp. 330–338.
  42. Princeton University, “About WordNet,” WordNet. Princeton University., 2010. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alsudais-2018-image-classification-arabic-assessing-accuracy/