Elamur Research⌕
☰
home / papers / 2023 / Enhancing Arabic Machine Translation for E-commerce Product Information: Data Quality Challenges and Innovative Selection Approaches

Enhancing Arabic Machine Translation for E-commerce Product Information: Data Quality Challenges and Innovative Selection Approaches

Paper · 2023 · Bryan Zhang, Salah Danial and Stephan Walter

Summary

The paper addresses noisy training data for English-Arabic machine translation of e-commerce product content (titles, descriptions, bulletpoints). Arabic is morphologically rich and has many dialects, so industrial systems pull large volumes of parallel data from many sellers and suppliers and must refresh it periodically. Manually validating each new batch is impractical. The authors identify two recurring defect types. The first is one source string mapped to many targets (or many sources to one target), which mixes translations, transliterations, dialect variants and outright errors. The second is text in the wrong language or in mixed languages. They propose two data selection methods. The heuristic one combines a cap on how many counterparts a segment may have with a filter that checks what share of characters belong to the expected script. The topic-based one clusters the corpus with a Dirichlet Multinomial Mixture and keeps only the large, clean clusters, which catches noise patterns nobody has seen before. Using both methods together gave the best results. Gains appeared in automatic metrics and human ratings, and a live A/B test in the Saudi store was positive. This shows that data curation alone, without changes to the model, can noticeably improve production MT.

Key points

  • Combined selection (HEU+TOPIC) vs. a baseline trained on the latest in-domain data:
    • Average gain across the three domains: +4.47% BLEU and +2.35% chrF.
    • Titles gained the most: +9.32% BLEU and +3.83% chrF.
    • Descriptions were essentially flat on BLEU (−0.02%) but rose +0.95% on chrF.
  • Heuristic filter configuration: the 1:M/N:1 filter used M = N = 5, chosen from a separate comparison. The script filter discarded anything not in English or Arabic.
  • Clustering method: DMM (Nigam et al., 2000) with Collapsed Gibbs Sampling (Yin and Wang, 2014), applied to symbolic text representations. The number of clusters is inferred automatically rather than fixed in advance.
  • Human and live evaluation:
    • Human raters judged hundreds of outputs and found fluency up 3.1% and adequacy up 3.29%.
    • A 4-week online A/B test in the KSA store favoured the new model.
  • Target variety: Modern Standard Arabic is the default output. Some regional adaptation helps, for example an Egyptian dialect word for "case" in the Egypt store.

Interesting ideas

  • I1. Treat an unusually high fan-out of counterparts per segment (one source with many targets, or the reverse) as a cheap noise signal; it needs no model and scales to any corpus size.
  • I2. Detect the wrong language by the fraction of characters that fall inside an expected alphabet, with a loose threshold so that only clearly foreign strings are removed and precision stays high.
  • I3. Cluster the training pool by surface patterns and keep the big clusters; small, rare clusters tend to hold the defects, including kinds of noise no hand-written rule anticipated.
  • I4. Report findings about bad clusters back to the data providers so that quality improves upstream, not just through filtering on the consumer side.
  • I5. Stack the filters, applying topic selection first and heuristics second, and measure each alone and combined per domain; the gains differed sharply between titles and descriptions.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when building silver data, flag sentences where witness parsers or LLM teachers produce many divergent analyses, and drop them as a group rather than voting.
  • I2 (tokens): run such a ratio check before the en tokenizer and tagger to route mixed-script or non-English fragments away instead of forcing English morphology onto them.
  • I3 (learn): cluster incoming silver sentences or self-induced repair cases, inspect the small clusters as likely error sources, and exclude them before training the parsers.
  • I4 (tree): when a treebank or UD dialect conversion yields small clusters of odd trees, record them as explainable conversion-bug reports that can be fixed at the source.
  • I5 (learn): evaluate each data-cleaning step separately and per genre (tests, word problems, fables), so that a step helping one reader is not assumed to help all.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,105 worksSemitic languages · 3,066 works

Cite

Cite as Bryan Zhang, Salah Danial and Stephan Walter. 2023. Enhancing Arabic Machine Translation for E-commerce Product Information: Data Quality Challenges and Innovative Selection Approaches. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.13

@inproceedings{zhang2023enhancing,
  title = {{Enhancing Arabic Machine Translation for E-commerce Product Information: Data Quality Challenges and Innovative Selection Approaches}},
  author = {Bryan Zhang, Salah Danial and Stephan Walter},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.13},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.13}
}

References

34 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Manar Alkhatib and Khaled Shaalan. 2018. The key challenges for arabic machine translation. Intelligent Natural Language Processing: Trends and Applications, pages 139–156. ↗
  2. Mohamed Seghir Hadj Ameur, Farid Meziane, and Ahmed Guessoum. 2020. Arabic machine translation: A survey of the latest trends and challenges. Computer Science Review, 38:100305. ↗
  3. Tianchi Bi, Liang Yao, Baosong Yang, Haibo Zhang, Weihua Luo, and Boxing Chen. 2020. Constraint translation candidates: A bridge between neural query translation and cross-lingual information retrieval. ↗
  4. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.". find
  5. Ingwer Borg and Patrick JF Groenen. 2005. Modern multidimensional scaling: Theory and applications. Springer Science & Business Media. find
  6. Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2019. Neural machine translation with sentence-level topic context. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(12):1970–1984. ↗
  7. Tobias Domhan, Michael Denkowski, David Vilar, Xing Niu, Felix Hieber, and Kenneth Heafield. 2020. The sockeye 2 neural machine translation toolkit at AMTA 2020. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 110– 115, Virtual. Association for Machine Translation in the Americas. ↗
  8. Vladimir Eidelman, Jordan Boyd-Graber, and Philip Resnik. 2012. Topic models for dynamic translation model adaptation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 115– 119. find
  9. B. Fuglede and F. Topsoe. 2004. Jensen-shannon divergence and hilbert space embedding. In International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings., pages 31–. find
  10. Jyoti Guha and Carmen Heger. 2014. Machine translation for global e-commerce on ebay. In Proceedings of the AMTA, volume 2, pages 31–37. find
  11. Nizar Habash. 2010. Introduction to arabic natural language processing. In Introduction to Arabic Natural Language Processing. ↗
  12. Yuening Hu, Ke Zhai, Vladimir Edelman, and Jordan Boyd-Graber. 2013. Topic models for translation domain adaptation. In Topic Models: Computation, Application, and Evaluation. NIPS Workshop. find
  13. Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, and Lingjun Zhao. 2020. Cross-lingual information retrieval with BERT. In Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020), pages 26–31, Marseille, France. European Language Resources Association. ↗
  14. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016a. Fasttext.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651. ↗
  15. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016b. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. ↗
  16. Mike Lowndes and Aditya Vasudevan. 2021. Market guide for digital commerce search. find
  17. Prashant Mathur, Marcello Federico, Selçuk Köprü, Sharam Khadivi, and Hassan Sawaf. 2015. Topic adaptation for machine translation of e-commerce content. In Proceedings of Machine Translation Summit XV: Papers, Miami, USA. find
  18. Carson Sievert and Kenneth Shirley. 2014. LDAvis: A method for visualizing and interpreting topics. In Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces, pages 63–70, Baltimore, Maryland, USA. Association for Computational Linguistics. ↗
  19. Marlies van der Wees, Arianna Bisazza, and Christof Monz. 2017. Dynamic data selection for neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1400–1410, Copenhagen, Denmark. Association for Computational Linguistics.
  20. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.
  21. Tasnim Mohiuddin, Philipp Koehn, Vishrav Chaudhary, James Cross, Shruti Bhosale, and Shafiq Joty. 2022. Data selection curriculum for neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1569– 1582, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  22. Haifeng Wang, Hua Wu, Zhongjun He, Liang Huang, and Kenneth Ward Church. 2021. Progress in machine translation. Engineering. ↗
  23. Shuyo Nakatani. 2010. Language detection library for java. find
  24. Deyi Xiong, Min Zhang, and Xing Wang. 2015. Topic-based coherence modeling for statistical machine translation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(3):483–493. ↗
  25. Jian-Yun Nie. 2010. Cross-language information retrieval. Synthesis Lectures on Human Language Technologies, 3(1):1–125. ↗
  26. Kamal Nigam, Andrew Kachites McCallum, Sebastian Thrun, and Tom Mitchell. 2000. Text classification from labeled and unlabeled documents using em. Machine learning, 39(2-3):103–134. ↗
  27. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. ↗
  28. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. ↗
  29. Andy Way. 2013. Traditional and emerging use-cases for machine translation. Proceedings of Translating and the Computer, 35:12. find
  30. Jianhua Yin and Jianyong Wang. 2014. A dirichlet multinomial mixture model-based approach for short text clustering. In SIGKDD, pages 233–242. ACM. find
  31. Jian Zhang, Liangyou Li, Andy Way, and Qun Liu. 2016. Topic-informed neural machine translation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1807–1817. find
  32. Mingyang Zhou, Runxiang Cheng, Yong Jae Lee, and Zhou Yu. 2018. A visual attention grounding neural model for multimodal machine translation. CoRR, abs/1808.08266. ↗
  33. Andreas Rücklé, Krishnkant Swarnkar, and Iryna Gurevych. 2019. Improved cross-lingual question retrieval for community question answering. In The World Wide Web Conference, WWW ’19, page 3179–3186, New York, NY, USA. Association for Computing Machinery. find
  34. Shadi Saleh and Pavel Pecina. 2020. Document translation vs. query translation for cross-lingual information retrieval in the medical domain. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6849– 6860, Online. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2023-enhancing-arabic-machine-translation-e/