Elamur Research⌕
☰
home / papers / 2026 / TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla

TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla

Summary

Bangla has more than 242 million native speakers, yet LLMs handle programming requests written in it poorly. The main reason is the lack of Bangla coding data for training and of benchmarks for evaluation. The authors address this with TigerCoder, the first family of Code LLMs built specifically for Bangla, in 1B and 9B sizes. They assembled about 300K Bangla instruction–code pairs from three sources: self-instruct generation, synthetic generation and machine translation of English material. They also built MBPP-Bangla, a human-translated and expert-checked version of MBPP with reference solutions in five programming languages. Before training, they tested two things. First, existing code models lose much of their accuracy when the task description is in Bangla rather than English. Second, translating Bangla prompts into English before generation gives similar or worse results. TigerCoder improves Pass@1 by 11–18% over multilingual and general-purpose Bangla LLMs. The work suggests that carefully curated data can let small open models serve low-resource languages in specialised domains.

Key points

  • Training data: three corpora of 100K examples each, totalling about 300K instruction–code pairs:
    • Bangla-Code-Instruct-SI (self-instruct)
    • Bangla-Code-Instruct-Syn (synthetic)
    • Bangla-Code-Instruct-TE (machine-translated)
  • MBPP-Bangla benchmark:
    • Contains 974 problems taken from MBPP.
    • Each problem has reference code in Python, Java, JavaScript, Ruby and C++, the original MBPP test cases and one of five topic labels (String, Math, Data-structures, Algorithms, File-I/O).
    • Two native speakers (TOEFL > 100) translated the problems independently, and one polyglot programmer verified them.
  • RQ1 (Bangla vs. English prompts): most models drop sharply when prompted in Bangla. On mHumanEval Pass@1:
    • GPT-3.5 falls from 0.79 to 0.56.
    • Phi-4 falls from 0.79 to 0.10.
    • TigerLLM is the main exception, staying close to its English scores.
  • RQ2 (translating prompts to English first): this step gives no reliable gain and often produces similar or lower Pass@k than prompting directly in Bangla.
  • Released resources: the TigerCoder models (e.g., TigerCoder-9B) and MBPP-Bangla are published on Hugging Face. The authors present the work as an extension of the TigerLLM line into software engineering, with an emphasis on reproducibility.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5nampb63np3m

@inproceedings{raihan2026tigercoder,
  title = {{TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla}},
  author = {Raihan, Nishat and Anastasopoulos, Antonios and Zampieri, Marcos},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5nampb63np3m},
  url = {https://doi.org/10.63317/5nampb63np3m}
}

References

44 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kabir Ahuja, Anirudh Das, Sandipan Das, Ashwini Deshpande, Sebastian Gehrmann, Anup Gopinath, Arya Guha, Pooja Kumar-Jois, Prem Mani, Ashwin Paranjape, et al. 2023. Mega: Multilingual evaluation of generative ai. arXiv preprint arXiv:2310.10567. find our card
  2. Kabir Ahuja, Karan Sikka, Madhusudana Nallasamy, and Chandrika Singh. 2024. Few-shot learning for low-resource languages with large language models. In Proceedings of the 62nd 7 https://lrec2026.info/calls/second-call-for-papers/ ↗
  3. Annual Meeting of the Association for Computational Linguistics. find
  4. Anthropic. 2023. Claude: The anthropic ai language model. Online documentation. Available at: https://www.anthropic.com. ↗
  5. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. In arXiv preprint arXiv:2108.07732. find our card
  6. Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M Sohel Rahman, and Rifat Shahriyar. 2022. Banglabert: Language model pretraining and benchmarks for low-resource language understanding evaluation in bangla. In Findings of the Association for Compu… find
  7. Pramit Bhattacharyya, Joydeep Mondal, Subhadip Maji, and Arnab Bhattacharya. 2023. Vacaspati: A diverse corpus of bangla literature. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Comp… find
  8. Damián Blasi, Antonios Anastasopoulos, and Graham Neubig. 2022. Systematic inequalities in language technology performance across the world’s languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). find our card
  9. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. In arXiv preprint arXiv:2107.03374. find our card
  10. Common Crawl Foundation. 2008–2025. Common crawl. https://commoncrawl.org. ↗
  11. Marta R Costa-Jussà, James Cross, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Adam Afflerbach, Adriane Abramowitz, Aditya Singhal, Aditya Balaji, Akshat Agrawal, Akshat Ravinuthala, Akshay Ramakrishnan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  13. Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md Sajid Altaf, Mohammed Saidul Islam, Tareq Mahmood Jamil, Shadman Sakib Alam, Irfan Kabir, Mohammad Nasim, Enamul Hossain, and Nawshad Akhter. 2022. Banglarqa: A benchmark dataset for under-resourced bangla language reading comprehension-based questio… find
  14. Google Gemma Team. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  15. Google. 2024. Gemini 2.5: The next generation of multimodal ai models. find
  16. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, et al. 2022. Training compute-optimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems. find
  17. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. find our card
  18. Sai Iyer, Jiawei Baek, Daisy Li, Zoekook, and Spencer Weiss. 2022. Optimizing tokenization for low-resource languages in machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. find
  19. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the nlp world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. find our card
  20. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. find
  21. Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2024. Quantifying multilingual performance of large language models across languages. arXiv preprint arXiv:2404.11553. find
  22. Microsoft. 2024. Phi-4: A 14b-parameter model with instruct-following and multi-modal capabilities. find
  23. Niklas Muennighoff, Alex Wang, Alena Fenogenova, Fangyu Huang, Francesca Toni, Adina Williams, and Colin Wang. 2023. Cross-lingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. find
  24. OpenAI. 2023. Gpt-4 technical report. preprint arXiv:2303.08774. arXiv OpenAI. 2024. Gpt-4o system card. find our card
  25. Liliana Pasquale, Antonino Sabetta, Marcelo d’Amorim, Péter Hegedűs, Mehdi Tarrit Mirakhorli, Hamed Okhravi, Mathias Payer, Awais Rashid, Joanna CS Santos, Jonathan M Spring, et al. 2025. Challenges to using large language models in code generation and repair. IEEE Security & Privacy, 23(2):81–88. find
  26. Nishat Raihan, Antonios Anastasopoulos, and Marcos Zampieri. 2025a. mHumanEval - a multilingual benchmark to evaluate large language models for code generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Langua… find
  27. Nishat Raihan, Mohammed Latif Siddiq, Joanna CS Santos, and Marcos Zampieri. 2025b. Large language models in computer science education: A systematic literature review. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, pages 938–944. find
  28. Nishat Raihan and Marcos Zampieri. 2025. Tiger-LLM - a family of bangla large language models. In Proceedings of ACL. find
  29. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702. find our card
  30. Pritika Rohera, Chaitrali Ginimav, Gayatri Sawant, and Raviraj Joshi. 2025. Better to ask in english? evaluating factual accuracy of multilingual llms in english and low-resource languages. arXiv preprint arXiv:2504.20022. — [arXiv:2504.20022](https://arxiv.org/abs/2504.20022) find
  31. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Iliç, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. find
  32. Lisa Schut, Yarin Gal, and Sebastian Farquhar. 2025. Do multilingual llms think in english? arXiv preprint arXiv:2502.15603. — [arXiv:2502.15603](https://arxiv.org/abs/2502.15603) find
  33. Sheikh Shafayat, H Hasan, Minhajur Mahim, Rifki Putri, James Thorne, and Alice Oh. 2024. BEnQA: A question answering benchmark for Bengali and English. In Findings of the Association for Computational Linguistics: ACL 2024. find
  34. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  35. Christopher Toukmaji and Jeffrey Flanigan. 2025. Prompt, translate, fine-tune, re-initialize, or instruction-tune? adapting llms for in-context learning in low-resource languages. In Proceedings of the ACL GEM Workshop. ArXiv:2506.19187. find
  36. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. find
  37. Md Nafis Uddin, Masum Khan, Nabila Hasan, and Mahmudul Hossain. 2023. Exploring code-mixed bangla text in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. find
  38. Tianyi Wang, Yang Ye, Panupong Pasupat, Aohan Wan, Grant Friedman, Jiacheng Tu, Maya Schaar, Jason Wei, Suriya Gunasekar, Matthew Richardson, et al. 2023a. Babelcode: Llm as a polyglot programmer. arXiv preprint arXiv:2303.03845. — [arXiv:2303.03845](https://arxiv.org/abs/2303.03845) find
  39. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, et al. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. find
  40. Yuhan Wang, Xuanhe Zhou, Ruoxi Chen, et al. 2023c. A comprehensive capability analysis of gpt-3 and gpt-3.5 series models. find
  41. Canwen Xu, Ruqing Wang, Yeyun Gong, et al. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244. find our card
  42. Xiang Yue, Yueqi Song, Akari Asai, et al. 2024. Pangea: A fully open multilingual multimodal llm for 39 languages. In The Thirteenth International Conference on Learning Representations. find
  43. Abdullah Khan Zehady, Safi Al Mamun, Naymul Islam, and Santu Karmaker. 2024. Bongllama: Llama for bangla language. arXiv preprint arXiv:2410.21200. — [arXiv:2410.21200](https://arxiv.org/abs/2410.21200) find
  44. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/raihan-2026-tigercoder-novel-suite-llms-code/