Elamur Research⌕
☰
home / papers / 2026 / Translation-CoT: A Human-Inspired Chain-of-Thought Framework for Multilingual LLM Translation

Translation-CoT: A Human-Inspired Chain-of-Thought Framework for Multilingual LLM Translation

Summary

LLM-based machine translation still suffers from mistranslations, hallucinations and stilted output, and these problems are worst for low-resource languages. The authors propose Translation-CoT, a single structured prompt modeled on how professional translators work. The model first prepares by extracting keyword equivalents (lexical retrieval), parsing grammatical structure and identifying the topic. It then drafts a literal translation, refines fluency and tone, and resolves idioms and ambiguities before giving a final output. No dictionaries or curated examples are needed.

The method was tested on FLORES-101 for 14 languages, one from each of 14 language families, in 28 English-centric directions. It was compared against zero-shot, in-context learning, ToT and LOT prompting. Translation-CoT generally scores higher on BLEU, ChrF and METEOR, especially for English→X and low-resource languages, but the gains are not uniform. Human evaluation for English⇔Bangla found fewer errors. The study shows that task-specific reasoning prompts can improve multilingual LLM translation without any training, though the approach does not replace dedicated MT systems.

Key points

  • Models tested: GPT-4o, GPT-4o-mini, LLaMA 3.1 and Gemma 2. GPT-4o performed best overall and outperformed NLLB on a number of better-resourced languages, among them Estonian, Korean, Bangla, Filipino and Azerbaijani.
  • Automatic metrics: Translation-CoT achieved the best summarized BLEU for 8 of the 14 languages. Zero-shot or ICL remained competitive for the other six, depending on model, direction and metric.
  • Human evaluation: For English⇔Bangla, Translation-CoT received higher preference scores, lower MQM penalties and fewer hallucinations.
  • External systems: Translation-CoT surpasses NLLB in some directions but stays below Google Translate in every available comparison.
  • Limitations:
    • Sentence-level, English-centric evaluation only.
    • No component ablations.
    • No neural metrics such as COMET or MetricX.
    • Possible FLORES-101 contamination in pretraining data.
    • Results come from single deterministic runs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{prama2026translation,
  title = {{Translation-CoT: A Human-Inspired Chain-of-Thought Framework for Multilingual LLM Translation}},
  author = {Prama, Tabia Tanzin and Lovato, Juniper L and Danforth, Chris and Dodds, Peter},
  booktitle = {Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)},
  year = {2026},
  url = {https://aclanthology.org/2026.amta-research.2/}
}

References

68 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dai, Y., Chan, C. F., Wong, Y. K., and Pun, T. H. (2025). Learning to Learn: When Chain-Of-Thought in Domain-Specific Machine Translation. 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8. find
  2. Agrawal, S., Zhou, C., Lewis, M., Zettlemoyer, L., and Ghazvininejad, M. (2022). In-context Examples Selection for Machine Translation. In Annual Meeting of the Association for Computational Linguistics. find our card
  3. Alves, D. M., Guerreiro, N. M., Alves, J., Pombal, J. P., Rei, R., de Souza, J. G. C., Colombo, P., and Martins, A. (2023). Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning. In Conference on Empirical Methods in Natural Language Processing. find
  4. Baker, M. (1992). In Other Words: A Coursebook on Translation. find
  5. Banerjee, S. and Lavie, A. (2005). find
  6. METEOR: An Auto-Dong, Q., Li, L., Dai, D., Zheng, C., Wu, Z., Chang, B., Sun, X., Xu, J., Li, L., and Sui, Z. (2022). A Survey on In-context Learning. In Conference on Empirical Methods in Natural Language Processing. find our card
  7. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A. S., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Rozière, B., Biron, B., T… find
  8. Filippova, K. (2020). Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data. In Findings. find
  9. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76:378– 382. find
  10. Freitag, M., Foster, G. F., Grangier, D., Ratnakar, V., Tan, Q., and Macherey, W. (2021). Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation. Transactions of the Association for Computational Linguistics, 9:1460–1474. find our card
  11. García, X., Bansal, Y., Cherry, C., Foster, G. F., Krikun, M., Feng, F., Johnson, M., and Firat, O. (2023). The unreasonable effectiveness of few-shot learning for machine translation. ArXiv, abs/2302.01398. — [arXiv:2302.01398](https://arxiv.org/abs/2302.01398) find
  12. Ghazvininejad, M., Gonen, H., and Zettlemoyer, L. (2023). Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation. ArXiv, abs/2302.07856. find
  13. Gile, D. (1995). Basic Concepts and Models for Interpreter and Translator Training. find
  14. Hatim, B. A. and Munday, J. (2005). Translation: An Advanced Resource Book. find
  15. He, Z., Liang, T., Jiao, W., Zhang, Z., Yang, Y., Wang, R., Tu, Z., Shi, S., and Wang, X. (2023). Exploring Human-Like Translation Strategy with Large Language Models. Transactions of the Association for Computational Linguistics, 12:229–246. find our card
  16. Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y. J., Afify, M., and Awadalla, H. H. (2023). How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. ArXiv, abs/2302.09210. find
  17. Iroda, K. and Risolat, I. (2025). Identifying Grammatical Issues in Translation: Methods and Prospects for Technical Education Students. SHS Web of Conferences. find
  18. Jiao, W., Wang, W., Huang, J.-T., Wang, X., and Tu, Z. (2023). Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine. find our card
  19. Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F. B., Wattenberg, M., Corrado, G. S., Hughes, M., and Dean, J. (2016). Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. Transactions of the Association for Computational L… find
  20. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of Tricks for Efficient Text Classification. ArXiv, abs/1607.01759. find
  21. Karpinska, M. and Iyyer, M. (2023). Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors Persist. In Conference on Machine Translation. find our card
  22. Goyal, N., Gao, C., Chaudhary, V., Chen, P.-J., Wenzek, G., Ju, D., Krishnan, S., Ranzato, M., Guzmán, F., and Fan, A. (2021). The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation. Transactions of the Association for Computational Linguistics, 10:522–538. find
  23. Koehn, P. (2009). A process study of computer-aided translation. Machine Translation, 23:241–263. find
  24. Guerreiro, N. M., Alves, D. M., Waldendorf, J., Haddow, B., Birch, A., Colombo, P., and Martins, A. (2023). Hallucinations in Large Multilingual Translation Models. Transactions of the Association for Computational Linguistics, 11:1500–1517. find our card
  25. Li, J., Zhou, H., Huang, S., Chen, S., and Chen, J. (2023). Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions. Transactions of the Association for Computational Linguistics, 12:576–592. find
  26. Li, J., Zhang, Z., and Zhao, H. (2022). Self-Prompting Large Language Models for Open-Domain QA. ArXiv, abs/2212.08635. — [arXiv:2212.08635](https://arxiv.org/abs/2212.08635) find
  27. Proceedings of the 17th Conference of the Association for Machine Translation in the Americas, Québec City, Canada, August 31 - September 2, 2026. Volume 1: Research Track Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., and Zettlemoyer, L. (2020). Multilingual Denoising… find
  28. Lommel, A. (2013). Multidimensional Quality Metrics : A Flexible System for Assessing Translation Quality. find our card
  29. Newmark, P. (1988). A textbook of translation. find
  30. Nguyen, L. and Xu, Y. (2025). Reasoning for Translation: Comparative Analysis of Chain-of-Thought and Tree-of-Thought Prompting for LLM Translation. In Annual Meeting of the Association for Computational Linguistics. find
  31. Oni, M. K. and Prama, T. T. (2025). Transformer-Based Low-Resource Language Translation: A Study on Standard Bengali to Sylheti. ArXiv, abs/2510.18898. — [arXiv:2510.18898](https://arxiv.org/abs/2510.18898) find
  32. OpenAI, J. A., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., ing Bao, H., Bavarian, M., Belgum, J., Bello, I., Berdine, J., Bernadett-Shapiro, G., Berner, C., Bogd… find our card
  33. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: a Method for Automatic Evaluation of Machine Translation. In Annual Meeting of the Association for Computational Linguistics. find our card
  34. Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., Ouyang, Y., and Tao, D. (2023). Towards Making the Most of ChatGPT for Machine Translation. In Conference on Empirical Methods in Natural Language Processing. find our card
  35. Pfau, M. (2015). Memes of Translation: The spread of ideas in translation theory. find
  36. Popovic, M. (2015). chrF: character n-gram F-score for automatic MT evaluation. In WMT@EMNLP. find our card
  37. Proceedings of the 17th Conference of the Association for Machine Translation in the Americas, Québec City, Canada, August 31 - September 2, 2026. Volume 1: Research Track Post, M. (2018). A Call for Clarity in Reporting BLEU Scores. In Conference on Machine Translation. find our card
  38. Prama, T. T. and Anwar, M. M. (2025). Sylheti to Standard Bangla Neural Machine Translation: A Deep Learning-Based Dialect Conversion Approach. In Bajaj, A., Madureira, A. M., and Abraham, A., editors, Hybrid Intelligent Systems. find
  39. HIS 2023. Lecture Notes in Networks and Systems, volume 1224. Springer, Cham. find
  40. Prama, T. T., Danforth, C. M., and Dodds, P. S. (2025a). BanglaMATH : A Bangla benchmark dataset for testing LLM mathematical reasoning at grades 6, 7, and 8. ArXiv, abs/2510.12836. find
  41. Prama, T. T., Danforth, C. M., and Dodds, P. S. (2025b). LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheti. ArXiv, abs/2511.21761. find
  42. Raffel, C., Shazeer, N. M., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2019). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res., 21:140:1–140:67. find
  43. Raunak, V., Menezes, A., Post, M., and Awadallah, H. H. (2023). Do GPTs Produce Less Literal Translations? ArXiv, abs/2305.16806. find
  44. Ren, S., Wu, Y., Liu, S., Zhou, M., and Ma, S. (2020). A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation. In Annual Meeting of the Association for Computational Linguistics. find
  45. Riviere, G. T. M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram’e, A., Ferret, J., Liu, P., Tafti, P. D., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C. L., Jerome, S., Tsitsulin, A., Vieillard, N., Stańczyk, P., Girgin, S., Momchev, N… find
  46. Robinson, N. R., Ogayo, P., Mortensen, D. R., and Neubig, G. (2023). ChatGPT MT: Competitive for High- (but Not Low-) Resource Languages. ArXiv, abs/2309.07423. find our card
  47. Schwenk, H., Wenzek, G., Edunov, S., Grave, E., and Joulin, A. (2019). CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web. In Annual Meeting of the Association for Computational Linguistics. find our card
  48. Taia.io (2025). 9 Reasons The Language Translation Process Is So Hard. team, N., Costa-jussà, M. R., Cross, J., cCelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., Sun, A., Wang, S., Wenzek, G., Youngblood, A., Akula, B., Barrault, L., Gonzalez, G.… find
  49. Terryn, A. R. and de Lhoneux, M. (2024). Exploratory Study on the Impact of English Bias of Generative Large Language Models in Dutch and French. In HUMEVAL. find
  50. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. (2023). LLaMA: Open and Efficient Foundation Language Models. ArXiv, abs/2302.13971. find
  51. Vilar, D., Freitag, M., Cherry, C., Luo, J., Ratnakar, V., and Foster, G. F. (2022). Prompting PaLM for Translation: Assessing Strategies and Performance. ArXiv, abs/2211.09102. find
  52. Wang, H., Wu, H., He, Z., Huang, L., and Church, K. W. (2021). Progress in Machine Translation. Engineering. find
  53. Wang, S., Yang, C.-H. H., Wu, J., and Zhang, C. (2023a). Can Whisper Perform Speech-Based In-Context Learning? ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 13421–13425. find
  54. L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y., editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational L… find
  55. Yang, A., Zhang, B., Hui, B., Gao, B., Yu, B., Li, C., Liu, D., Tu, J., Zhou, J., Lin, J., Lu, K., Xue, M., Lin, R., Liu, T., Ren, X., and Zhang, Z. (2024). Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement. ArXiv, abs/2409.12122. find our card
  56. Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. (2023). Large Language Models as Optimizers. ArXiv, abs/2309.03409. find our card
  57. Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., and Jiang, M. (2022). Generate rather than Retrieve: Large Language Models are Strong Context Generators. ArXiv, abs/2209.10063. find our card
  58. Zhang, J. and Zong, C. (2020). Neural machine translation: Challenges, progress and future. Science China Technological Sciences, 63:2028–2050. find
  59. Zhang, Z., Zhang, A., Li, M., and Smola, A. J. (2022). Automatic Chain of Thought Prompting in Large Language Models. ArXiv, abs/2210.03493. find our card
  60. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E. H., and Zhou, D. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. ArXiv, abs/2203.11171. find our card
  61. Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., and Smola, A. J. (2023). Multimodal Chain-of-Thought Reasoning in Language Models. find our card
  62. Trans. Mach. Learn. Res., 2024. find
  63. Wang, Y., Zhang, Z., and Wang, R. (2023b). Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method. In Annual Meeting of the Association for Computational Linguistics. find
  64. Zhu, D., Chen, J., Haydarov, K., Shen, X., Zhang, W., and Elhoseiny, M. (2023a). ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions. ArXiv, abs/2303.06594. — [arXiv:2303.06594](https://arxiv.org/abs/2303.06594) find
  65. Weaver, W. (1955). Rededication to Liberal Decency. Science, 121 3138:15A. find
  66. Zhu, W., Liu, H., Dong, Q., Xu, J., Kong, L., Chen, J., Li, L., and Huang, S. (2023b). Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. ArXiv, abs/2304.04675. find our card
  67. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E. H., Xia, F., Le, Q., and Zhou, D. (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. ArXiv, abs/2201.11903. find our card
  68. Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2021). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. In Toutanova, K., Rumshisky, A., Zettlemoyer, Proceedings of the 17th Conference of the Association for Machine Translation i… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/prama-2026-translation-cot-human-inspired-chain/