Elamur Research⌕
☰
home / papers / 2025 / Niyamika at BHASHA Task 1: Word-Level Transliteration for English-Hindi Mixed Text in Grammar Correction Using MT5

Niyamika at BHASHA Task 1: Word-Level Transliteration for English-Hindi Mixed Text in Grammar Correction Using MT5

Summary

The authors took part in the IndicGEC 2025 shared task at the BHASHA workshop. The task was to correct grammatical errors in sentences in five Indian languages: Hindi, Bengali, Malayalam, Tamil and Telugu. The system works in two steps. First, IndicTrans transliterates English or romanized words into the script of the main language, one word at a time. Then mT5-small corrects the grammar. The official training data were very scarce, so the initial GLEU scores were low. The authors therefore took 10,000 short sentences per language from IndicCorpV2 and artificially introduced errors into 70% of them by inserting, deleting or swapping characters or words. The final model was trained on Hindi only. It was applied to the other languages without fine-tuning, in a zero-shot setting. On the test data, GLEU rose to 79–90, and the best result was third place for Telugu. The paper shows that synthetic data augmentation helps when a language has few resources. The effect of transliteration was barely visible here, because the test data contained very few English words.

Key points

  • The official IndicGEC 2025 corpus is very small: from 91 training pairs for Tamil to 659 for Bengali. The baseline GLEU on test ranged from 0.46 for Tamil to 27.07 for Malayalam.
  • For augmentation, 10,000 sentences of 5–15 words each were selected per language from IndicCorpV2, and 7,000 of them were corrupted. The six error types in the Hindi corpus were distributed almost evenly, at roughly 16–17% each.
  • Final GLEU on test: Hindi 79.47, Bengali 81.83, Malayalam 89.77, Tamil 84.48, Telugu 85.03. Leaderboard positions: 9/11, 7/8, 7/8, 9/9 and 3/8, respectively.
  • Transliteration gave a small gain only for Hindi: 79.47 versus 78.98 without it. The errors found include incorrect transliteration of rare names such as CHATGPT. In the other languages, Devanagari admixtures and changes in sentence meaning occurred.
  • The model was trained with the following parameters: mT5-small, learning rate 2e-4, 21 epochs, maximum sequence length 128. The code is available on GitHub in the Rucha-Ambaliya/bhasha-workshop repository.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.bhasha-1.13

@inproceedings{ambaliya2025niyamika,
  title = {{Niyamika at BHASHA Task 1: Word-Level Transliteration for English-Hindi Mixed Text in Grammar Correction Using MT5}},
  author = {Ambaliya, Rucha and Dugar, Mahika and Mishra, Pruthwik},
  booktitle = {Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian Languages (BHASHA 2025)},
  year = {2025},
  doi = {10.18653/v1/2025.bhasha-1.13},
  url = {https://doi.org/10.18653/v1/2025.bhasha-1.13}
}

References

18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Irshad Ahmad Bhat, Vandan Mujadia, Aniruddha Tam-mewar, Riyaz Ahmad Bhat, and Manish Shrivastava. 2015. IIIT-H System Submission for FIRE2014 Shared Task on Transliterated Search. In Proceedings of the Forum for Information Retrieval Evaluation (FIRE ’14), pages 48–53. find
  2. Numerical normalization: English numerals were not converted into their corresponding Indic script representations (e.g., 123 → १२३), which could affect readability and grammatical correctness in certain contexts. find
  3. Monojit Choudhury, Gokul Chittaranjan, Parth Gupta, and Amitava Das. 2014. Overview of fire 2014 track on transliterated search. find
  4. Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023. Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages. In Proceedings of ACL 2023, pages 12402–12426. find our card
  5. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891, Dublin, Ireland. Association for Computation… find our card
  6. Md. Akmal Hassan, Md. Saiful Islam, and Mumit Khan. 2016. Sentiment Analysis on Bangla and Romanized Bangla Text Using Long Short-Term Memory Recurrent Neural Network. arXiv preprint arXiv:1610.00369. — [arXiv:1610.00369](https://arxiv.org/abs/1610.00369) find
  7. Braja Gopal Patra, Dipankar Das, and Amitava Das. 2015. Shared Task on Sentiment Analysis in Indian Languages (SAIL) at MIKE 2015. In Proceedings of the International Conference on Mining Intelligence and Knowledge Exploration (MIKE), pages 650–655. find
  8. Shashank Sharma, Srinivas Pykl, and Chandra Rakesh. 2015. Text normalization of code mix and sentiment analysis. In 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 1468–1473. find
  9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  10. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. In Proceedings of NAACL-HLT 2021, pages 483–498. find
  11. Aditya Joshi, Ameya Prabhu, Manish Shrivastava, and Vasudeva Varma. 2016. Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text. In Proceedings of COL-ING 2016, pages 2482–2491. find
  12. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In Proceedings of the International Conference on Learning Representations. find
  13. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. 2018. Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task. In Proceedings of NAACL-HLT 2018, pages 595–606. find
  14. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Associ… find our card
  15. Anoop Kunchukuttan, Siddharth Jain, and Rahul Kejri-wal. 2021. A large-scale evaluation of neural machine transliteration for indic languages. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3469–3475. find
  16. Yash Madhani, Sushane Parthan, Priyanka Bedekar, Gokul Nc, Ruchi Khapra, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh M Khapra. 2023. Aksharantar: Open indic-language transliteration datasets and models for the next billion users. In Findings of the association for computational linguistics: Emnlp… find
  17. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground Truth for Grammatical Error Correction Metrics. In Proceedings of ACL-IJCNLP 2015, pages 588–593. find
  18. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of CoNLL-2014, pages 1–14. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ambaliya-2025-niyamika-bhasha-task-1-word/