Elamur Research⌕
☰
home / papers / 2026 / When Words Don't Mean What They Say: Figurative Understanding in Bengali Idioms

When Words Don't Mean What They Say: Figurative Understanding in Bengali Idioms

Summary

Large language models still struggle to interpret idioms, whose intended sense differs from their literal wording, and the problem is sharper for under-resourced languages such as Bengali. Earlier Bengali work covered only small idiom sets, general multi-word expressions or metaphors (e.g., Alankaar). The authors address this with the Bangla Bagdhara dataset, a collection of 10,361 Bengali idioms. Each idiom is described by a 19-field JSON schema that records literal and figurative readings in Bangla and English, English counterparts, example sentences, usage domain, sentiment, flags for historical, religious and cultural significance, and socio-spatial setting. A six-person team gathered entries from sources covering about three centuries over five months. AI tools produced draft translations and sentiment labels, which experts then corrected and agreed on through consensus. To test models, 30 multilingual and instruction-tuned LLMs were asked to infer the meanings of 100 idioms selected by the experts. No model reached 50% accuracy, while human participants averaged 83.4%. The released resource gives a basis for culturally aware Bengali NLP, including translation and educational tools.

Key points

  • Size and comparison: The corpus has 10,361 idioms with 19 annotation fields. It is larger than the other Bengali and Konkani resources cited: Alankaar has 2,500 items with 2 fields, and Konidioms has 1,597 items with 9 fields. MAGPIE (50,000+) is larger, but it is English-only.
  • Schema groups: The 19 fields fall into four groups:
    • semantic and cross-lingual mapping;
    • pragmatic context;
    • sociocultural and affective metadata, including Boolean significance flags and sentiment;
    • diachronic and ethnolinguistic information (the scape, history and note fields).
  • Annotation team: One computer scientist and five specialists in Bangla language and linguistics built the corpus. They validated entries by deliberative consensus rather than individual scoring rubrics.
  • Benchmark results: 30 LLMs were tested on 100 idioms, and none exceeded 50% accuracy. The human baseline came from 15 Bangladeshi students (6 high school, 9 undergraduate), who scored a mean of 83.4%.
  • Availability: The dataset is publicly released on Kaggle (sakhadib/bangla-bagdhara), and the authors describe it as an ongoing, expanding project.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/546w2cys6m6t

@inproceedings{sakhawat2026when,
  title = {{When Words Don't Mean What They Say: Figurative Understanding in Bengali Idioms}},
  author = {Sakhawat, Adib and Parveen, Shamim Ara and Amin, Md Ruhul and Khatun, Tahera and Mahmud, Shamim Al and Islam, Md Saiful},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/546w2cys6m6t},
  url = {https://doi.org/10.63317/546w2cys6m6t}
}

References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ruchit Agrawal, Vighnesh Chenthil Kumar, Vigneshwaran Muralidharan, and Dipti Sharma. 2018. No more beating about the bush : A step towards idiom handling for Indian language NLP. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japa… find
  2. Aryaman Arora, Adam Farris, Samopriya Basu, and Suresh Kolichala. 2022. Computational historical linguistics and language diversity in South Asia. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1396–1409, Dublin, Ireland. Ass… find
  3. Paul Baker, Andrew Hardie, Tony McEnery, and B. D. Jayaram. 2003. Constructing corpora of south asian languages. find
  4. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022. FLUTE: Figurative language understanding through textual explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7139–7159, Abu Dhabi, United Arab Emirates. Associat… find our card
  5. Tanmoy Chakraborty, Dipankar Das, and Sivaji Bandyopadhyay. 2014. Identifying bengali multiword expressions using semantic clustering. find
  6. Ashwini Deo and Devyani Sharma. 2006. Typological variation in the ergative morphology of Indo-Aryan languages. Final draft, to appear in Linguistic Typology. find
  7. Michael Flor, Xinyi Liu, and Anna Feldman. 2025. A survey of idiom datasets for psycholinguistic and computational research. In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Long and Short Papers, pages 90–100, Hannover, Germany. HsH Applied Academics. find our card
  8. Hessel Haagsma, Johan Bos, and Malvina Nissim. 2020. MAGPIE: A large corpus of potentially idiomatic expressions. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 279–287, Marseille, France. European Language Resources Association. find our card
  9. Ayesha Khatun, Md Gulzar Hussain, Md Islam, Sumaiya Kabir, and Md Mahin. 2020. An emperical framework of idioms translator from bengali to english: Rule based approach. find
  10. Paria Khoshtab, Danial Namazifard, Mostafa Masoudi, Ali Akhgary, Samin Mahdizadeh Sani, and Yadollah Yaghoobzadeh. 2025. Comparative study of multilingual idioms and similes in large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 8680–8698, A… find our card
  11. Diego Moussallem, Mohamed Ahmed Sherif, Diego Esteves, Marcos Zampieri, and Axel-Cyrille Ngonga Ngomo. 2018. LIdioms: A multilingual linked idioms data set. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Re… find
  12. Partha Pakray, Alexander Gelbukh, and Sivaji Bandyopadhyay. 2025. Natural language processing applications for low-resource languages. Natural Language Processing, 31(2):183–197. find
  13. Geetanjali Rakshit and Jeffrey Flanigan. 2025. Alankaar: A dataset for figurativeness understanding in bangla. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI era, pages 998–1002, Varna, Bulgaria… find
  14. Naziya Mahamdul Shaikh, Jyoti D. Pawar, and Mubarak Banu Sayed. 2024. Konidioms corpus: A dataset of idioms in Konkani language. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9932–9940, Torino, Ital… find
  15. Caroline Sporleder, Linlin Li, Philip Gorinski, and Xaver Koch. 2010. Idioms in context: The IDIX corpus. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  16. Laura Street, Nathan Michalov, Rachel Silverstein, Michael Reynolds, Lurdes Ruela, Felicia Flowers, Angela Talucci, Priscilla Pereira, Gabriella Morgon, Samantha Siegel, Marci Barousse, Antequa Anderson, Tashom Carroll, and Anna Feldman. 2010. Like finding a needle in a haystack: Annotating the Amer… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sakhawat-2026-when-words-don-t-mean/