Elamur Research⌕
☰
home / papers / 2023 / Findings of the WMT 2023 Shared Task on Low-Resource Indic Language Translation

Findings of the WMT 2023 Shared Task on Low-Resource Indic Language Translation

Summary

This report covers the WMT 2023 shared task on machine translation for low-resource languages of Northeast India. These languages have little digital text, and that shortage makes MT hard to build. Teams built systems translating between English and Assamese, Mizo, Khasi and Manipuri, in both directions. The organizers released IndicNE-Corp1.0, which combines parallel and monolingual corpora from their earlier projects. Sources include the Bible, the Xobdo and Glosbe dictionaries, PMIndia, news sites and blogs. Each team could enter one PRIMARY system per direction, trained without extra parallel data, plus up to two CONTRASTIVE runs. Outputs were scored with BLEU, TER, RIBES, COMET and ChrF, and experts also rated samples by hand. The best systems mostly fine-tuned pretrained multilingual models, such as IndicBART and IndicTrans2, or used denoising and data augmentation. Translating into English worked better than translating out of it, because the Indic languages are morphologically rich. The task matters because it provides a shared benchmark for languages from four language families that MT research has largely ignored.

Key points

  • Parallel data in IndicNE-Corp1.0 (train/validation/test):
    • en–as: 50,000 / 2,000 / 2,000
    • en–lus: 50,000 / 1,500 / 2,000
    • en–kha: 24,000 / 1,000 / 1,000
    • en–mni: 21,687 / 1,000 / 1,000 (Manipuri in Bengali script)
  • Monolingual data: about 2.62M Assamese sentences, 1.91M Mizo, 2.14M Manipuri and only 182,737 Khasi.
  • Participation: 31 teams registered, but only 9 submitted system description papers. Results from teams without a paper were left out of the report.
  • Top BLEU by pair:
    • IACS-LRILT (IndicBART): both en–as directions and mni→en
    • NICT-AI4B (IndicTrans2): both en–lus directions
    • IOL Research (T5/BART denoising plus augmentation): both en–kha directions
    • CUNI: en→mni
  • Human evaluation: experts rated 20 sentences per primary submission on a 0–5 scale for adequacy, fluency and an overall score. Fluency consistently scored higher than adequacy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 29 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.wmt-1.56

@inproceedings{pal2023findings,
  title = {{Findings of the WMT 2023 Shared Task on Low-Resource Indic Language Translation}},
  author = {Pal, Santanu and Pakray, Partha and Laskar, Sahinur Rahman and Laitonjam, Lenin and Khenglawt, Vanlalmuansangi and Warjri, Sunita and Dadure, Pankaj Kundan and Dash, Sandeep Kumar},
  booktitle = {Proceedings of the Eighth Conference on Machine Translation},
  year = {2023},
  doi = {10.18653/v1/2023.wmt-1.56},
  url = {https://doi.org/10.18653/v1/2023.wmt-1.56}
}

References

6 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Goutam Agrawal, Rituraj Das, Anupam Biswas, and Dalton Meitei Thounaojam. 2023. Neural machine translation for english - manipuri and english - assamese. find
  2. Mazida Ahmed, Kuwali Talukdar, Parvez Boruah, Shikhar Kumar Sarma, and Kishore Kashyap. 2023. Guit-nlp’s submission to shared task: Low resource indic language translation. find
  3. Rupjyoti Baruah, Rajesh Kumar Mundotiya, and Anil Kumar Singh. 2021. Low resource neural machine translation: Assamese to/from other indo-aryan (indic) languages. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 21(1). find
  4. Raj Dabre, Jay Gala, and Pranjal Chitale. 2023. Nict-ai4b’s submission to the indic mt shared task in wmt 2023. find
  5. Raj Dabre, Himani Shrotriya, Anoop Kunchukuttan, Ratish Puduppully, Mitesh Khapra, and Pratyush Kumar. 2022. IndicBART: A pre-trained model for indic natural language generation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1849–1863, Dublin, Ireland. Association for… find
  6. Team Name IACS-LRILT CFILT-IITB NICT-AI4B IOL Research CUNI Organizer MUNI-NLP ATULYA-NITS GUIT-NLP Translation Type English-To-Assamese (Primary) English-To-Assamese (Contrastive-1) English-To-Assamese (Contrastive-2) Assamese-To-English (Primary) Assamese-To-English (Contrastive-1) Assamese-To-Eng… find

This page in Markdown → https://md.research.elamur.ai/free/papers/pal-2023-findings-wmt-2023-shared-task/