Elamur Research⌕
☰
home / papers / 2024 / ALADAN at IWSLT24 Low-resource Arabic Dialectal Speech Translation Task

ALADAN at IWSLT24 Low-resource Arabic Dialectal Speech Translation Task

Summary

The paper describes the ALADAN team's entry to the IWSLT 2024 dialectal, low-resource track, which translates Levantine (apc) and Tunisian (aeb) Arabic speech into English. These dialects have little training data and no fixed spelling, so the same word appears in many written forms. The authors built a cascade:

  • ASR: TDNNF and Zipformer recognisers, combined.
  • MT: NLLB-200 1.3B distilled and CohereAI's Command-R, the latter fine-tuned with QLoRA.
  • Data: extra crowdsourced and multidialectal sets such as PADIC and MADAR.

Their main contribution is a reusable normalization method for dialectal Arabic text. A Word2Vec model first finds words that occur in similar contexts. A Levenshtein distance with Arabic-specific edit costs then keeps only candidates that look like spelling variants. Humans review the resulting clusters.

Normalization improved ASR by up to 18%. The best submission merged many Command-R checkpoints through MBR decoding plus a genetic algorithm and reached 31.5 BLEU on the apc validation set. The work matters because it shows that cleaning inconsistent spelling, together with pooling data across dialects, pays off when resources are scarce.

Key points

  • Edit costs: semivowels and Alif cost 0.1 to insert or delete, while other consonants cost 1.5. Substitutions between Alif variants, or between interchangeable consonant pairs such as emphatics, cost 0.3.
  • Variant search: a CBOW Word2Vec model (100 dimensions, window 5) proposes the 50 nearest neighbours above cosine 0.3. Candidates are kept if their weighted distance is below 3, and the largest clusters are checked by hand.
  • ASR results: the combined system reached WER 23.6 on apc validation, 32.7 on aeb dev and 33.1 on aeb test1.
  • Fine-tuning setup: Command-R was tuned in 4-bit with QLoRA (r of 8, 16 or 32) for 5000 updates on one A100. Varying the hyperparameters changed little.
  • Final systems: the primary submission combined 26 checkpoints and scored 31.5 BLEU on ASR input. Command-R was the only LLM that produced usable output with a context-aware prompt.

Interesting ideas

  • I1. Edit costs can encode linguistic knowledge: cheap edits for vowel or variant letters and expensive ones for consonant changes separate true spelling variants from different words. — For Mova (tokens): a weighted distance with cheap vowel or doubling edits could map noisy or variant English spellings to lemmas without merging distinct words.
  • I2. Using distributional similarity to shrink the candidate set before an expensive string comparison makes variant discovery tractable, and leaves only clusters for human review. — For Mova (learn): self-induced repair rules for unknown tokens could be proposed this way and then confirmed by witness parsers or a teacher.
  • I3. The authors deliberately rejected a popular aggressive normalization because it merged words with different meanings, preferring targeted fixes. — For Mova (tokens): normalization rules should be checked against minimal pairs, as a negative control, to make sure they never collapse distinct lemmas.
  • I4. Letters placed where they can only end a word, appearing mid-token, served as a cheap signal for glued compounds and typos. — For Mova (tokens): impossible character positions, such as a capital mid-word or punctuation patterns, could flag run-together tokens for splitting before tagging.
  • I5. Combining many checkpoints through MBR plus a genetic search gave better output than any single model. — For Mova (back): candidate generations from the meaning graph could be chosen by consensus scoring across several realisations rather than taking a single one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.iwslt-1.25

@inproceedings{kheder2024aladan,
  title = {{ALADAN at IWSLT24 Low-resource Arabic Dialectal Speech Translation Task}},
  author = {Kheder, Waad Ben and Jon, Josef and Beyer, André and Messaoudi, Abdel and Affan, Rabea and Barras, Claude and Tychonov, Maxim and Gauvain, Jean-Luc},
  booktitle = {Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024)},
  year = {2024},
  doi = {10.18653/v1/2024.iwslt-1.25},
  url = {https://doi.org/10.18653/v1/2024.iwslt-1.25}
}

References

49 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt4 technical re port. arXiv preprint arXiv:2303.08774. ↗ our card
  2. Sejnowski. 1985. A learning algorithm for boltz mann machines. Cogn. Sci., 9:147169. find
  3. Milind Agarwal, Sweta Agarwal, Antonios Anasta sopoulos, Luisa Bentivogli, Ondřej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cet tolo, Mingda Chen, et al. 2023. Findings of the iwslt 2023 evaluation campaign. Association for Compu tational Linguistics. find our card
  4. Antonios Anastasopoulos, Loc Barrault, Luisa Ben tivogli, Marcely Zanon Boito, Ondřej Bojar, Roldano Cattoni, Anna Currey, Georgiana Dinu, Kevin Duh, Maha Elbayad, et al. 2022. Findings of the iwslt 2022 evaluation campaign. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 98157. Association for Computational Linguistics. find our card
  5. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A frame work for selfsupervised learning of speech represen tations. Advances in neural information processing systems, 33:1244912460. find
  6. Marcely Zanon Boito, John Ortega, Hugo Riguidel, ̈ Antoine Laurent, Loı c Barrault, Fethi Bougares, Fi ras Chaabani, Ha Nguyen, Florentin Barbier, Souhir Gahbiche, et al. 2022. Ontrac consortium systems for the iwslt 2022 dialect and lowresource speech translation tasks. arXiv preprint arXiv:2205.01987. ↗
  7. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdul rahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Confer ence on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Re sources Association (ELRA). find
  8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are fewshot learners. Advances in neural information processing systems, 33:18771901. find
  9. ChungCheng Chiu, Arun Narayanan, Wei Han, Rohit Prabhavalkar, Yu Zhang, Navdeep Jaitly, Ruoming Pang, Tara N Sainath, Patrick Nguyen, Liangliang Cao, et al. 2021. Rnnt models fail to generalize to outofdomain audio: Causes and solutions. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 873880. IEEE. find
  10. Marta R Costajussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling humancentered machine translation. arXiv preprint arXiv:2207.04672. ↗
  11. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Preprint, arXiv:2305.14314. ↗
  12. Jonathan G Fiscus. 1997. A postprocessing system to yield reduced word error rates: Recognizer output voting error reduction (rover). In 1997 IEEE Work shop on Automatic Speech Recognition and Under standing Proceedings, pages 347354. IEEE. find
  13. Jennifer Drexler Fox, Desh Raj, Natalie Delworth, Quinn McNamara, Corey Miller, and Migüel Jetté. 2024. Updated corpora and benchmarks for longform speech recognition. In ICASSP 2024 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1324613250. IEEE. find
  14. Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2021. Minimum bayes risk decoding with neural metrics of translation quality. find
  15. Pegah Ghahremani, Vimal Manohar, Hossein Hadian, Daniel Povey, and Sanjeev Khudanpur. 2017. In vestigation of transfer learning for asr using lfmmi trained neural networks. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 279286. IEEE. find
  16. Vaibhava Goel and William J Byrne. 2000. Minimum bayesrisk automatic speech recognition. Computer Speech & Language, 14(2):115135. find
  17. Alex Graves. 2012. Sequence transduction with recur rent neural networks. Preprint, arXiv:1211.3711. ↗ our card
  18. Anmol Gulati, James Qin, ChungCheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. 2020. Conformer: Convolutionaugmented trans former for speech recognition. arXiv preprint arXiv:2005.08100. ↗ our card
  19. Nizar Habash, Mona T Diab, and Owen Rambow. 2012. Conventional orthography for dialectal arabic. In LREC, pages 711718. find our card
  20. Soufiane Hajbi, Omayma Amezian, Nawfal El Moukhi, Redouan Korchiyne, and Younes Chihab. 2024. Mo roccan arabizitoarabic conversion using rulebased transliteration and weighted levenshtein algorithm. Scientific African, 23:e02073. find
  21. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. Preprint, arXiv:1503.02531. ↗ our card
  22. WeiNing Hsu, Benjamin Bolte, YaoHung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdel rahman Mohamed. 2021. Hubert: Selfsupervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:34513460. find
  23. Amir Hussein, Cihan Xiao, Neha Verma, Thomas The baud, Matthew Wiesner, and Sanjeev Khudanpur. 2023. Jhu iwslt 2023 dialect speech translation sys tem description. In Proceedings of the 20th Interna tional Conference on Spoken Language Translation (IWSLT 2023), pages 283290. find
  24. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mis tral 7b. arXiv preprint arXiv:2310.06825. ↗
  25. Josef Jon and Ondřej Bojar. 2023. Breeding machine translations: Evolutionary approach to survive and thrive in the world of automated evaluation. In Pro ceedings of the 61st Annual Meeting of the Associa tion for Computational Linguistics (Volume 1: Long Papers), pages 21912212, Toronto, Canada. Asso ciation for Computational Linguistics. find
  26. Josef Jon, Martin Popel, and Ondřej Bojar. 2023. CUNI at WMT23 general translation task: MT and a ge netic algorithm. In Proceedings of the Eighth Con ference on Machine Translation, pages 119127, Singapore. Association for Computational Linguis tics. find
  27. Mateusz Krubiński, Hashem Sellat, Shadi Saleh, Adam Pospíšil, Petr Zemánek, and Pavel Pecina. 2023. Multiparallel corpus of north levantine arabic. In Proceedings of ArabicNLP 2023, pages 411417. find
  28. Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally typical sampling. Transac tions of the Association for Computational Linguis tics, 11:102121. find our card
  29. Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. ↗
  30. Tomas Mikolov, Kai Chen, Greg Corrado, and Jef frey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  31. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor rado, and Jeff Dean. 2013b. Distributed representa tions of words and phrases and their compositional ity. Advances in neural information processing sys tems, 26. find
  32. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, ZhengXin Yong, Hai ley Schoelkopf, et al. 2022. Crosslingual general ization through multitask finetuning. arXiv preprint arXiv:2211.01786. ↗ our card
  33. Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A. ChoquetteChoo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. Madlad 400: A multilingual and documentlevel large au dited dataset. Preprint, arXiv:2309.04662. ↗ our card
  34. Kishore Papineni, Salim Roukos, Todd Ward, and Wei Jing Zhu. 2002. Bleu: a method for automatic eval uation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Com putational Linguistics, pages 311318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find
  35. Shankar Kumar and William Byrne. 2004. Minimum Bayesrisk decoding for statistical machine transla tion. In Proceedings of the Human Language Tech nology Conference of the North American Chapter of the Association for Computational Linguistics: HLTNAACL 2004, pages 169176, Boston, Mas sachusetts, USA. Association for Computational Lin guistics. find
  36. Yifan Peng, Siddharth Dalmia, Ian Lane, and Shinji Watanabe. 2022. Branchformer: Parallel mlp attention architectures to capture local and global context for speech recognition and understanding. In International Conference on Machine Learning, pages 1762717643. PMLR. find
  37. Teven Le Scao, Angela Fan, Christopher Akiki, El lie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2023. Bloom: A 176b parameter openaccess multilingual language model. find
  38. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre training for neural machine translation. Transac tions of the Association for Computational Linguis tics, 8:726742. find
  39. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. Machine translation experiments on padic: A parallel arabic dialect corpus. In Proceedings of the 29th Pacific Maja Popović. 2015. chrF: character ngram Fscore for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392395, Lisbon, Portugal. Association for Computational Linguistics. find
  40. Daniel Povey, Gaofeng Cheng, Yiming Wang, Ke Li, Hainan Xu, Mahsa Yarmohammadi, and Sanjeev Khudanpur. 2018. Semiorthogonal lowrank ma trix factorization for deep neural networks. In Inter speech, pages 37433747. find
  41. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Mar tins. 2022. COMET22: UnbabelIST 2022 submis sion for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578585, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  42. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Alham Fikri Aji, Zhengzhong Liu, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Preslav Nakov, Timothy Bald win, and Eric Xing. 2023. Jais and jaischat: Arabiccentric foundation and instructiontuned open generative large language models. Preprint, arXiv:2308.16149. ↗ our card
  43. Yixuan Su and Nigel Collier. 2022. Contrastive search is what you need for neural text generation. arXiv preprint arXiv:2210.14140. ↗
  44. Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Ling peng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. In Advances in Neural Information Processing Systems. find
  45. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, MarieAnne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi cient foundation language models. arXiv preprint arXiv:2302.13971. ↗
  46. Ioannis Tsiamas, Gerard I Gállego, Carlos Escolano, José Fonollosa, and Marta R Costajussà. 2022. Pre trained speech encoders and efficient finetuning methods for speech translation: Upc at iwslt 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 265276. find
  47. Brian Yan, Patrick Fernandes, Siddharth Dalmia, Jia tong Shi, Yifan Peng, Dan Berrebbi, Xinyi Wang, Graham Neubig, and Shinji Watanabe. 2022. Cmu’s iwslt 2022 dialect speech translation system. In Pro ceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 298307. find
  48. Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2023. Zipformer: A faster and bet ter encoder for automatic speech recognition. arXiv preprint arXiv:2310.11230. ↗
  49. Ahmet Üstün, Viraat Aryabumi, ZhengXin Yong, WeiYin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, HuiLee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Long pre, Niklas Muennighoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. Aya model: An instruction finetuned openaccess multilingual lan guage model. arXiv preprint arXiv:2402.07827. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kheder-2024-aladan-iwslt24-low-resource-arabic/