Elamur Research⌕
☰
home / papers / 2026 / Cascaded Modular or End-to-End? : An Investigation on Speech-to-Speech Translation Task for Dravidian Languages

Cascaded Modular or End-to-End? : An Investigation on Speech-to-Speech Translation Task for Dravidian Languages

Paper · 2026 · Bhavana Nali and Abhik Jana

Summary

Speech-to-speech translation for Dravidian languages has seen little systematic study, mainly because parallel speech data are scarce. This workshop paper compares two design choices for Tamil, Telugu and Kannada: a cascaded modular pipeline and a single End-to-End model. The cascade chains a recognizer with IndicTrans2 for text translation and Indic Parler-TTS for synthesis. The recognizer is Whisper Large-v3 for English input and IndicConformer for Dravidian input. SeamlessM4T-v2 serves as the End-to-End counterpart. Both systems are tested zero-shot and after LoRA adaptation on FLEURS and the Mann-ki-Baat portion of BhasaAnuvaad, in English→Dravidian and Dravidian→Dravidian directions. Mann-ki-Baat has no direct Dravidian pairs, so the authors build them automatically with LaBSE similarity and Hungarian matching. Evaluation uses BLEU, chrF++ and COMET, plus WER/CER from re-transcribing the generated audio. The cascade wins in most configurations, though some gaps are under one BLEU point and no significance tests were run. LoRA tuning brings clear gains, especially between Dravidian languages. The work is a first joint benchmark of both paradigms for these languages, and the code is public.

Key points

  • BLEU ranges: the cascade scores 3.17–18.96 zero-shot and 5.08–19.18 fine-tuned. SeamlessM4T-v2 scores 3.02–15.72 and 4.11–16.84. The best result is EN→Telugu on FLEURS (19.18 vs 16.84 after tuning).
  • LoRA setup: rank 16, scaling 32, dropout 0.1, with under 1% of parameters trainable. Training ran 4,000 steps with batch size 8 and learning rate 1e-4. Indic Parler-TTS stayed frozen.
  • Data: about 3,000 FLEURS utterances per language. Mann-ki-Baat yields roughly 15,284 aligned pairs (similarity threshold 0.6), split 80/10/10.
  • Dravidian→Dravidian gains: Ta→Te rises from 8.13 to 10.12 and Ta→Kn from 6.73 to 8.74. Error propagation is visible for EN→Tamil on FLEURS: MT BLEU is 17.42 with gold transcripts but 14.81 with ASR input. English ASR WER falls from 0.124 to 0.089.
  • Caveats:
    • Using IndicConformer for back-ASR may slightly favour the cascade.
    • There was no human evaluation of naturalness.
    • The automatic alignments were not validated.
    • Fine-tuning did not always help: cascaded EN→Kannada on Mann-ki-Baat dropped from 8.70 to 7.03 BLEU.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Indian Institute of Technology Bhubaneswar.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Bhavana Nali and Abhik Jana. 2026. Cascaded Modular or End-to-End? : An Investigation on Speech-to-Speech Translation Task for Dravidian Languages. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.3

@inproceedings{nali2026cascaded,
  title = {{Cascaded Modular or End-to-End? : An Investigation on Speech-to-Speech Translation Task for Dravidian Languages}},
  author = {Bhavana Nali and Abhik Jana},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.3},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.3}
}

References

8 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  2. Kaushal Santosh Bhogale, Deovrat Mehendale, Tahir Javed, Devbrat Anuragi, Sakshi Joshi, Sai Sundaresan, Aparna Ananthanarayanan, Sharmistha Dey, Anusha Srinivasan, Abhigyan Raman, Mitesh M. Khapra, and Anoop Kunchukuttan. 2025. Mahad-hwani: Bringing parity in speech recognition for low-resource indi… find
  3. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982– 3992,…
  4. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2022. Fleurs: Few-shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE.
  5. Ashwin Sankar, Sparsh Jain, Nikhil Narasimhan, Devilal Choudhary, Dhairya Suman, Mohammed Safi Ur Rahman Khan, Anoop Kunchukuttan, Mitesh M. Khapra, and Raj Dabre. 2025a. Towards building large scale datasets and state-of-the-art automatic speech translation systems for 14 indian languages. In Proce… find
  6. Jay Gala, Pranjal A Chitale, A K Raghavan, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar M, Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M Khapra, Raj Dabre, and Anoop Kunchukuttan. 2023. Indictrans2: Towards high-quality and accessible machine tran…
  7. Ashwin Sankar, Yoach Lacombe, Sherry Thomas, Praveen Srinivasa Varadhan, Sanchit Gandhi, and Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Mitesh M. Khapra. 2025b. Rasmalai: Resources for adaptive speech modeling in indian languages with accents and intonations. In Proceedings of the 26th Annual C… find
  8. Seamless Communication Team. 2023. Seamlessm4t: Massively multilingual and multimodal machine translation. arXiv preprint arXiv:2308.11596.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nali-2026-cascaded-modular-or-end-end/