Elamur Research⌕
☰
home / papers / 2026 / Building and Evaluating a High Quality Parallel Corpus for English Urdu Low Resource Machine Translation

Building and Evaluating a High Quality Parallel Corpus for English Urdu Low Resource Machine Translation

Summary

Machine translation between English and Urdu is held back by a lack of good parallel data. Existing public resources tend to cover narrow domains, are poorly aligned, or are too small. The authors address this by building a curated corpus of 80,749 English–Urdu sentence pairs spanning 18 domains. Their pipeline combines ethically sourced texts, structural and manual sentence alignment, deduplication and a strict length-based filter (AWCD ≤ 5). They then turn the corpus into a bidirectional SFT dataset with instructions in both English and Urdu, which doubles the number of examples and makes the models less sensitive to prompt language. Fine-tuning Llama-3.1-8B-Instruct and UrduLlama 1.1 on this data clearly beats the untuned baseline. Llama-FT is especially strong for Urdu→English and performs almost identically whichever prompt language is used. Blind human ratings support the automatic scores. The work offers a reusable recipe for building clean, domain-balanced corpora for other low-resource languages. However, testing so far uses only held-out data from the same corpus.

Key points

  • Corpus: 80,749 aligned English–Urdu pairs across 18 domains, filtered with AWCD ≤ 5. It leans heavily toward news, and the authors plan to release it publicly.
  • Llama-FT sacreBLEU scores: 24.65–25.24 for En→Ur and 76.14–77.97 for Ur→En. For Ur→En, the Base model scores about 17 and UrduLlama-FT about 29.
  • Bilingual instructions largely remove the effect of prompt language for Llama-FT. The Base model and UrduLlama-FT lose up to about 3 BLEU when prompted in Urdu.
  • Human evaluation: 90 sentences per direction, two bilingual annotators, 5-point Likert scale.
    • En→Ur: Base 1.05, UrduLlama-FT 3.80, Llama-FT 3.68.
    • Ur→En: Base 3.36, UrduLlama-FT 3.57, Llama-FT 3.86.
  • Limitations: evaluation covers only the corpus's own domains. Planned work includes tests on FLORES-200 and WAT Urdu tasks and applying the recipe to smaller base models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.loresmt-1.8

@inproceedings{tahir2026building,
  title = {{Building and Evaluating a High Quality Parallel Corpus for English Urdu Low Resource Machine Translation}},
  author = {Tahir, Munief Hassan and Azam, Hunain and Shams, Sana and Hussain, Sarmad},
  booktitle = {Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.loresmt-1.8},
  url = {https://doi.org/10.18653/v1/2026.loresmt-1.8}
}

References

18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahsan Ahmad and Mohsin Ali Ahmad. 2024. Advancing Roman Urdu to Urdu transliteration using machine learning techniques. Asian Journal of Multidisciplinary Research & Review, 5(2):108–127. find
  2. Syed Abdul Basit Andrabi and Abdul Wahid. 2022. Machine translation system using deep learning for English to Urdu. Computational Intelligence and Neuroscience, 2022:7873012. ↗
  3. Umer Butt, Stalin Varanasi, and Günter Neumann. 2025. Low-resource transliteration for Roman-Urdu and Urdu using transformer-based models. In Proceedings of the Eighth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2025), pages 144–153. Association for Computational Linguistics. find
  4. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783. ↗
  5. Muhammad Naeem Ul Hassan, Zhengtao Yu, Jian Wang, Ying Li, Shengxiang Gao, Shuwan Yang, and Cunli Mao. 2024. LKMT: Linguistics knowledge-driven multi-task neural machine translation for Urdu and English. Computers, Materials & Continua, 81(1):1901–1923. ↗
  6. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. ↗
  7. Huma Israr, Muhammad Khuram Shahzad, and Shahid Anwar. 2024. Improved Urdu–English neural machine translation with a fully convolutional neural network encoder. International Journal of Mathematical, Engineering and Management Sciences, 9(5):1067–1088. ↗
  8. Bushra Jawaid, Amir Kamran, and Ondřej Bojar. 2016. Enriching source for English-to-Urdu machine translation. In Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing (WSSANLP@COLING), pages 54–63, Osaka, Japan. Association for Computational Linguistics. find
  9. Ahmed Raza, Usama Ahmed, Kainat Saleem, Muhammad Azam Hussain, and Amna Sarwar. 2024. Urdu-to-english-based unsupervised machine translation. Journal of Computer Science and Applications, 1(2):1–10. ↗
  10. SIL International. 2022. Urdu. https://www.ethnologue.com/language/urd. Ethnologue: Languages of the World (25th edition, accessed 2025). torchtune maintainers and contributors. 2024. torchtune: Pytorch’s finetuning library. ↗
  11. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 13484–13508. ↗
  12. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Guoyin Wang, and 1 others. 2023. Instruction tuning for large language models: A survey. ACM Computing Surveys. find our card
  13. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024. Multilingual machine translation with large language models: Empirical results and analysis. In Findings of the association for computational linguistics: NAACL 2024, pages 2765–2781. find our card
  14. Shahnawaz Khan and Imran Usman. 2019. A model for English to Urdu and Hindi machine translation system using translation rules and artificial neural network. The International Arab Journal of Information Technology, 16(1):125–131. find
  15. Sajadul Hassan Kumhar, Syed Immamul Ansarullah, Akber Abid Gardezi, Shafiq Ahmad, Abdelaty Edrees Sayed, and Muhammad Shafiq. 2022. Translation of english language into urdu language using lstm model. Computers, Materials & Continua, 74(2):3899–3912. ↗
  16. Hafsa Masroor, Muhammad Saeed, Maryam Feroz, Kamran Ahsan, and Khawar Islam. 2019. Transtech: development of a novel translator for roman urdu to english. Heliyon, 5(5):e01780. ↗
  17. Usman Mohy ud Din. 2019. Urdu–English machine transliteration using neural networks. Master’s thesis, COMSATS University Islamabad, Lahore Campus. find
  18. Matt Post. 2018. A call for clarity in reporting bleu scores. Preprint, arXiv:1804.08771. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tahir-2026-building-evaluating-high-quality-parallel/