Elamur Research⌕
☰
home / papers / 2024 / Can ChatGPT capture swearing nuances? Evidence from translating Arabic oaths

Can ChatGPT capture swearing nuances? Evidence from translating Arabic oaths

Summary

The paper asks whether ChatGPT can render Arabic oath expressions into English while keeping their religious and cultural weight. The author treats oaths as performative speech acts that are tied to whatever the swearer holds sacred or precious, such as God, the Qur'an, the Prophet, one's children or one's health. Thirty Arabic oaths were gathered from earlier literature and translated with ChatGPT. Each output was then compared against a human reference translation. Rather than giving one quality score, the study sorts the outputs into a taxonomy of "gaps": religious, cultural, both religious and cultural, no gap, use of non-oath particles, redundancy, and failure to recover the diacritics Arabic script usually omits. The author concludes that ChatGPT still handles oaths poorly. The paper recommends using it only for a first draft, followed by human post-editing. It also argues that training data should include more Arabic oath usage and the rituals behind it. The work matters because it shows how culture-bound speech acts expose weaknesses that fluent surface output hides.

Key points

  • The data are 30 Arabic oath expressions taken from prior literature. ChatGPT's English versions were judged against human translations.
  • The error categories are religious gap, cultural gap, combined gap, no gap, non-oath particles, redundancy, and diacritic misreading.
  • ChatGPT read the verb in a request to swear as "ask," producing "I ask you by God!" where the human version is "Swear me by God!" It also produced wordier variants such as "the One who created you."
  • Diacritic errors:
    • An unvocalized word meaning "sake" in an oath was taken as "stone."
    • The hamza position on aliph decides whether the speaker or the hearer swears, and ChatGPT confused the two forms.
  • Particle errors:
    • Arabic oath particles are wa, bi-, fauq and ta-, while English uses by and on.
    • ChatGPT used "above" four times (e.g., "Above the Quran!"), "and" once, and the modal "may" twice.

Interesting ideas

  • I1. Diagnose translation failures with a typed set of gap categories instead of one score, so each class of error points to its own cause. — For Mova (learn): Evaluation reports could break failures into named classes (role lost, particle wrong, sense misread), making regressions traceable to specific pipeline stages.
  • I2. Model an oath as a speech act with distinct parts: who commits, what is sworn by, and whether it asserts or promises. This captures what a translation must preserve. — For Mova (frames): prag could have a "swear" speech-act frame with slots for the committing party, the sacred or valued entity and the backed proposition, so a later step can check which slots survived.
  • I3. A tiny written mark can flip which participant carries the obligation (speaker swears vs. hearer is asked to swear). Who is bound matters more than the words used. — For Mova (back): When translating through the meta-world, the generator should verify that the committed participant in the source graph is still the committed one in the output text.
  • I4. Each language has a small closed inventory of oath particles (English: by, on). Output can be checked against that list, which would catch errors like "above." — For Mova (back): Generation could keep per-construction lists of licensed markers and flag any realized oath or similar fixed construction that uses an unlicensed particle.
  • I5. Ambiguous unvocalized words get resolved by the surrounding construction: inside an oath, the reading "sake" fits and "stone" does not. — For Mova (tokens): Lemma and sense selection could take a hint from a detected construction frame, preferring the reading that fits the slot instead of the most frequent one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2412.02466 · arXiv 2412.02466

@misc{shormani2024can,
  title = {{Can ChatGPT capture swearing nuances? Evidence from translating Arabic oaths}},
  author = {Shormani, Mohammed Q.},
  year = {2024},
  doi = {10.48550/arxiv.2412.02466},
  eprint = {2412.02466},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.02466}
}

References

37 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alowedi, Noha & AlAhdal, Aref. 2023. Artificial Intelligence based Arabic-to-English machine versus human translation of poetry: An analytical study of outcomes. Journal of Namibian Studies 33(52). 1523-1538. https://doi.org/10.59670/jns.v33i.800 ↗
  2. Bassnett, Susan. 2014. Translation studies. 4th edition. find
  3. Routledge Brown, Kristen. 1999. Possible and Questionable: Opening Nietzsche's Genealogy to Feminine Body. Hypatia 14(3): 39-58. find
  4. Cascella, Marco, Jonathan Montomoli, Valentina Bellini & Elena Bignami. 2023. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios, J. Med. Syst. 47(1). 1–5. doi: 10.1007/s10916-023-01925-4 ↗
  5. Catford, John Cunnison. 1965. A Linguistic theory of translation: an essay in applied linguistics. Oxford: Oxford University Press. find
  6. Charlesworth, Max. 2012. Translating Religious Texts. Sophia. 51.423–448 Chomsky, Noam. 1965. Aspects of the theory of syntax. MIT Press. find
  7. Conklin, Blane W. 2005. Oath formulae in Classical Hebrew and other Semitic languages. Unpublished Ph.D Thesis, University of Chicago. de Almeida, Giselle and O’Brien, Sharon. 2010. Analysing post-editing performance: correlations with years of translation experience. In Proceedings of the 14th annual conference of the European association for machine translation, St. Raphaël, France, 27–28. find
  8. Dergaa, Ismail, Karim Chamari, Piotr Zmijewski, & Helmi Ben Saad. 2023. From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biol Sport. 40(2). 615–622. http://dx.doi.org/10.5114/biolsport.2023.125623 ↗
  9. Goodwin, Philip. 2013. Translating the English Bible from Relevance to Deconstruction. find
  10. James Clarke & C. Groves, Declan and Dag Schmidtke 2009. Identification and analysis of post-editing patterns for MT. In Proceedings of the twelfth machine translation Summit, August 26–30, Ottawa. 429–436. find
  11. Hendy, Amr, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, et al. 2023. How good are GPT models at machine translation? a comprehensive evaluation. Available from https://arxiv.org/abs/2302.09210 . (Accessed on September 3, 2023). ↗
  12. Jakobson, Roman. 1959. On Linguistic Aspects of Translation. In Brower Reuben Arthur (ed.), On Translation, 232-239. Harvard University Press. find
  13. Jay, Timothy. 2009. The utility and ubiquity of taboo words. Perspectives on Psychological Science 4(2). 153-161. http://dx.doi.org/10.1111/j.1745-6924.2009.01115.x ↗
  14. Jiao, Wenxiang, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi & Zhaopeng Tu. 2023. Is ChatGPT a good translator? Yes with GPT-4 as the engine. Available from https://arxiv.org/abs/2301.08745Z. (Accessed December 24, 2023). ↗ our card
  15. Kenny, Dorothy. 2022. Human and machine translation. In Dorothy Kenny (ed.), Machine translation for everyone: Empowering users in the age of artificial intelligence, 23–49. Berlin: Language Science Press. find
  16. Krings, Hans. 2001. Repairing texts: empirical investigations of machine translation post-editing processes. Kent, OH.: The Kent State University Press. find
  17. Kung, Tiffany H., Cheatham, Morgan., Medenilla, Arielle et al. 2023. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PloS Digital Health 1-12. https://doi.org/10.1371/journal.pdig.0000198 ↗
  18. Larroyed, Aline. 2023. Redefining Patent Translation: The Influence of ChatGPT and the Urgency to Align Patent Language Regimes in Europe with Progress in Translation Technology. GRUR International. 72(11): 1009–1017. https://doi.org/10.1093/grurint/ikad099 ↗
  19. Lee, Tong King. 2023. Artificial intelligence and Posthumanist translation: ChatGPT vs the translator. Applied Linguistics Review doi: 10.1515/applirev-2023-0122 ↗
  20. Newmark, Peter. 1981. Approaches to translation. Oxford: Oxford University press. find
  21. Newmark, Peter. 1988. A Textbook of translation. London: Prentice Hall. find
  22. Nida, Albert Eugene & Russell Taber Charles. 1969. The Theory and Practice of Translation. Brill. find
  23. Nida, Albert Eugene. 1964. Toward a science of translating with special reference to principles and procedures involved in Bible translating. Adler's Foreign Books Inc. find
  24. O’Brien, Sharon. 2007. An empirical investigation of temporal and technical post-editing effort. Translation and Interpreting Studies (TIS) 2(1). 83–136. find
  25. O’Brien, Sharon. 2011. Towards predicting post-editing productivity. Machine Translation 25:197–215. Peng, Baolin. find
  26. Chunyuan Li, Pengcheng He, Michel Galley & Jianfeng Gao. 2023. Instruction tuning with GPT-4. arXiv preprint arXiv:2304.03277. POV. ChatGPT: An AI NLP model. Available from https://www.ltimindtree.com/wp-content/uploads/2023/02/ChatGPT-An-AI-NLP-Model-POV.pdf ↗ our card
  27. Qarabesh, Mohammed, Mohammed Shormani & Arif Alahdal. 2023. (Un)translatability of (Yemeni) Arabic Oath Expressions into English. World Journal of English Language 13(3). 4-15. http://dx.doi.org/10.5430/wjel.v13n3p4 ↗
  28. Radford, Alec., Jeffrey Wu, Rewon Child, et al. 2019. Language models are unsupervised multitask learners. OpenAI. Available from: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf. (Accessed September 27, 2023). ↗
  29. Ray, Partha Pratim. 2023. Background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems 121-154.https://doi.org/10.1016/j.iotcps.2023.04.003 Sallam, Malik, Salim, Nesreen A., Al-Tammemi, Barakat Ala'a B., et al. 2023. ChatGPT output regarding compulsory vaccination and COVID-19 vaccine conspiracy: a descriptive study at the outset of a paradigm shift in online search for information. Cureus 15(2).e35029. doi:10.7759/cureus.35029.PMID ↗
  30. Schlesinger, Herbert J. 2008. Promises, oaths, and vows: on the psychology of promising. New York: Taylor & Francis Group. find
  31. Shormani, Mohammed. 2020. Does culture translate? Evidence from translating proverbs. Babel, John Benjamins 66(6), 902 – 927. find
  32. Sirriyya, Sawsan. 2009. Translating Islamic religious texts into English. Yarub House for Publishing and Distributing, Damascus. find
  33. Siu, Sai Cheong. 2023. ChatGPT and GPT-4 for professional translators: exploring the potential of large language models in translation. Preprint. 1-36. Available from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4448091 (Accessed on December 12, 2023). van Dis, Eva, Johan Bollen, Willem Zuidema, Robert van Rooij & Claudi L. Bockting. 2023. ChatGPT: five priorities for research. Nature 614. 224–226. http://dx.doi.org/10.1038/d41586-023-00288-7 ↗
  34. Vaswani, A., Noam Shazeer, Niki Parmar, et al. 2017. Attention is all you need. Available from https://arxiv.org/abs/1706.03762 (Accessed on 11 December, 2024) Xu, Draco. 2022. Learning in Biologically Plausible Neural Networks. Proceeding of 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Available from https://www.sas.rochester.edu/mth/undergraduate/honorspaperspdfs/d_xu23.pdf. ↗
  35. (Accessed on November 29, 2023). find
  36. Yogesh, Kumar, Apeksha Koul, Ruchi Singla & Muhammad Ijaz. 2023. Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda. Ambient Intell Humaniz Comput 4(7). 8459-8486. find
  37. Zheng, Binghan, Sergey Tyulenev, & Kobus Marais. 2023. Introduction: (re-)conceptualizing translation in translation studies. Translation Studies 16 (2): 167– 177. doi:10.1080/14781700.2023.2207577 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shormani-2024-chatgpt-capture-swearing-nuances-evidence/