Elamur Research⌕
☰
home / papers / 2026 / Few-shot Prompting or Supervised Tuning? A Comparative Study of LLMs for Linguistically Distant Language Pairs in BDI

Few-shot Prompting or Supervised Tuning? A Comparative Study of LLMs for Linguistically Distant Language Pairs in BDI

Summary

The paper examines bilingual dictionary induction (BDI) for language pairs that are typologically far apart and morphologically rich, a setting where mapping methods like VecMap and contrastive approaches tend to fail. The authors compared three ways of using LLMs. The first is an unsupervised setup that takes averaged last-layer token representations. The second is supervised contrastive fine-tuning following Li et al. (2022), with hard negatives drawn from VecMap nearest neighbours. The third is zero-shot and 5-shot prompting with templates from Li et al. (2023). Five models were tested: mT5 small, mT5 base, XGLM 564M, mGPT 1.3B and Llama-3.2 1B. The pairs pair English with Manipuri, Finnish, Turkish, Japanese and Tamil, plus English-Italian as a close-pair reference, evaluated in both translation directions. Five-shot prompting beat the unsupervised and zero-shot setups everywhere and beat fine-tuning in most cases, especially when translating into English. It still struggled when English was the source and the target was morphologically complex. The study suggests that prompting is a cheap, overfitting-resistant option when labelled dictionaries are scarce.

Key points

  • 5-shot prompting outperformed the unsupervised and zero-shot settings in every case and outperformed supervised fine-tuning in 82.86% of comparisons.
  • For Llama-3.2 1B, supervised fine-tuning did better than 5-shot prompting on En-Mn, En-Fi, En-Tr, En-Ja, Ja-En and En-Ta. The authors attribute this to limited training data for those languages, a bias toward English, and difficulty with morphology.
  • All models scored higher on the similar pair En-It than on the distant pairs. The unsupervised and zero-shot settings suffered most on Manipuri, Finnish, Turkish and Tamil.
  • Gains stop growing at about 5 shots, so adding more examples brings diminishing returns.
  • Error analysis used FinnWordNet, TurkishWordNet, TamilWordNet and a native Manipuri speaker. Typical errors were predicting a bare root or a wrongly inflected form instead of the correct affixed word.
  • The main stated limitation is that every pair includes English, so the findings may not carry over to pairs of two low-resource languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5aqjysq8avd3

@inproceedings{naorem2026few,
  title = {{Few-shot Prompting or Supervised Tuning? A Comparative Study of LLMs for Linguistically Distant Language Pairs in BDI}},
  author = {Naorem, Deepen and Singh, Sanasam Ranbir and Singh, Telem Joyson and Sarmah, Priyankoo},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5aqjysq8avd3},
  url = {https://doi.org/10.63317/5aqjysq8avd3}
}

References

44 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2016. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2289–2294, Austin, Texas. Association for Computati… find our card
  2. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. Learning bilingual word embeddings with (almost) no bilingual data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451–462, Vancouver, Canada. Association for Computational… find our card
  3. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018a. A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 789–798, Melbourne, Australi… find our card
  4. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018b. Unsupervised neural machine translation. In International Conference on Learning Representations. find our card
  5. Marta Bañón, Malina Chichirau, Miquel Esplà-Gomis, Mikel L. Forcada, Aarón Galiano-Jiménez, Cristian García-Romero, Taja Kuzman, Nikola Ljubešić, Rik van Noord, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Peter Rupnik, Vít Suchomel, Antonio Toral, and Jaume Zaragoza-Bernabeu. 2023. Turkish-english p… find
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  7. Zhoujun Cheng, Jungo Kasai, and Tao Yu. 2023. Batch prompting: Efficient inference with large language model APIs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 792–810, Singapore. Association for Computational Linguistics. find
  8. Santwana Chimalamarri, Dinkar Sitaram, and Ashritha Jain. 2020. Morphological segmentation to improve crosslingual word embeddings for low resource languages. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 19(5). find
  9. Sirajul Islam Choudhury, Leihaorambam Sarbajit Singh, Samir Borgohain, and Pradip Kumar Das. 2004. Morphological analyzer for manipuri: design and implementation. In Asian Applied Computing Conference, pages 123–129. Springer. find
  10. Paula Czarnowska, Sebastian Ruder, Ryan Cotterell, and Ann Copestake. 2020. Morphologically aware word-level translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2847–2860, Barcelona, Spain (Online). International Committee on Computational Linguistics… find
  11. Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, and Ann Copestake. 2019. Don’t forget the long tail! a comprehensive analysis of morphological generalization in bilingual lexicon induction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing an… find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  13. Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023. Towards leaving no Indic language behind: Building monolingual corpora, benchmark and models for Indic languages. In Proceedings of the 61st Annual Meeting of the Asso… find our card
  14. Abhimanyu Dubey et al. 2024. The llama 3 herd of models. find
  15. Kshitij Gupta, Benjamin Th’erien, Adam Ibrahim, Mats L. Richter, Quentin G. Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort. 2023. Continual pre-training of large language models: How to (re)warm your model? ArXiv, abs/2308.04014. — [arXiv:2308.04014](https://arxiv.org/abs/2308.04014) find
  16. Barry Haddow and Faheem Kirefu. 2020. PMIndia – A Collection of Parallel Corpora of Languages of India. arXiv e-prints, page arXiv:2001.09907. — [arXiv:2001.09907](https://arxiv.org/abs/2001.09907) find
  17. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers, pages 79–86. find
  18. Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya. 2018. The IIT Bombay English-Hindi parallel corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find our card
  19. Yaoyiran Li, Anna Korhonen, and Ivan Vulić. 2023. On bilingual lexicon induction with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9577–9599, Singapore. Association for Computational Linguistics. find
  20. Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, and Ivan Vulić. 2022. Improving word translation via two-stage contrastive learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4353–4374, Dublin, Ireland. Associatio… find
  21. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyano… find our card
  22. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput. Surv., 55(9). find our card
  23. Yuhan Liu, Xiuying Chen, Gao Xing, Ji Zhang, and Rui Yan. 2024. IAD: In-context learning ability decoupler of large language models in meta-training. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 85… find
  24. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. find
  25. Tomás Mikolov, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. CoRR, abs/1309.4168. find
  26. Deepen Naorem, Okram Jimmy Singh, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2023. English-manipuri cross-lingual embedding: A preliminary study. In 2023 International Conference on Asian Language Processing (IALP), pages 74–79. find
  27. Deepen Naorem, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2024a. Embarking on a preliminary exploration: Cross-lingual embedding in english-manipuri. International Journal of Asian Language Processing, 34(02):2450007. find
  28. Deepen Naorem, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2024b. Improving linear orthogonal mapping based cross-lingual representation using ridge regression and graph centrality. Computer Speech & Language, 87:101640. find
  29. Deepen Naorem, Sanasam Ranbir Singh, Telem Joyson Singh, and Priyankoo Sarmah. 2025. Mace: Morphology aware cross-lingual embedding using contrastive learning. IEEE Transactions on Audio, Speech and Language Processing, 33:3124–3136. find
  30. Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg, Matthew R. Gormley, and Graham Neubig. 2019. Bilingual lexicon induction with semi-supervision in non-isometric embedding spaces. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 184–193, Florence, I… find
  31. Reid Pryzant, Youngjoo Chung, Dan Jurafsky, and Denny Britz. 2018. JESC: Japanese-English subtitle corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  32. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. J. Artif. Int. Res., 65(1):569–630. find
  33. Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, Anastasia Kozlova, Vladislav Mikhailov, and Tatiana Shavrina. 2024. mgpt: Few-shot learners go multilingual. Transactions of the Association for Computational Linguistics, 12:58–79. find our card
  34. Thoudam Doren Singh and Savaji Bandyopadhyay. 2010a. Statistical machine translation of English-Manipuri using morpho-syntactic and semantic information. In Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Student Research Workshop, Denver, Colorado, USA.… find
  35. Thoudam Doren Singh and Sivaji Bandyopadhyay. 2010b. Manipuri-English bidirectional statistical machine translation systems using morphology and dependency relations. In Proceedings of the 4th Workshop on Syntax and Structure in Statistical Translation, pages 83–91, Beijing, China. find
  36. Coling 2010 Organizing Committee. find
  37. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu… find our card
  38. Y. Tang, C. Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. ArXiv, abs/2008.00401. find
  39. Ahmet Üstün, Gosse Bouma, and Gertjan van Noord. 2019. Cross-lingual word embeddings for morphologically rich languages. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 1222–1228, Varna, Bulgaria. INCOMA Ltd. find
  40. Ivan Vulić, Goran Glavaš, Roi Reichart, and Anna Korhonen. 2019. Do we really need fully unsupervised cross-lingual embeddings? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… find
  41. Ivan Vulić and Marie-Francine Moens. 2015. Monolingual and cross-lingual information retrieval models based on (bilingual) word embeddings. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 363–372. find
  42. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7222–7240, Online. Association for Computational… find our card
  43. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. find
  44. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naorem-2026-few-shot-prompting-or-supervised/