Elamur Research⌕
☰
home / papers / 2023 / Yet Another Model for Arabic Dialect Identification

Yet Another Model for Arabic Dialect Identification

Paper · 2023 · Ajinkya Kulkarni and Hanan Aldarmaki

Summary

The paper addresses spoken Arabic dialect identification (ADI). On the two standard benchmarks, ADI-5 (MGB-3) and ADI-17 (MGB-5), the winning challenge systems had never been beaten in later work. The authors limit their search to a 2×2 grid. The networks are ResNet and ECAPA-TDNN, both borrowed from speaker verification. The input features are either classic MFCCs or frozen embeddings from UniSpeech-SAT Large, a self-supervised model pretrained only on English speech. Each network ends with an AAM-softmax layer. At inference, its scores are blended with cosine similarity to a sampled cohort of training examples for each class. Training uses noise, reverberation and speed augmentation, plus a schedule that shortens the training segments partway through.

ECAPA-TDNN beats ResNet, and UniSpeech-SAT features beat MFCCs by roughly 4–5 accuracy points. An equal-weight fusion of all four variants sets new best results: 84.7% on ADI-5 and 96.9% on ADI-17. The study shows that self-supervised speech features transfer across languages to a fine-grained dialect task. It also hints that non-linguistic cues such as tone may help separate dialects.

Key points

  • Benchmarks.
    • ADI-5 has 5 classes (EGY, LAV, GLF, NOR, MSA) drawn from broadcast news, with 13,825 training utterances.
    • ADI-17 has about 3,000 hours of YouTube audio in 17 classes. Its training labels come from the video's country of origin. Its dev and test sets (about 33 and 25 hours) were checked by human annotators.
  • Accuracy gains. On ADI-5, fusion reaches 84.7% versus 75.0% for the earlier best (MIT-QCRI). On ADI-17, it reaches 96.9% versus 94.9% for the earlier best (DKU). ECAPA-TDNN with UniSpeech-SAT alone already scores 82.5% and 96.1%.
  • Features. UniSpeech-SAT Large is kept frozen and yields 1024-dimensional embeddings; it was pretrained on 94K hours of English. The MFCC baseline uses 80 dimensions with a 25 ms window and 10 ms step.
  • Classifier and inference.
    • The AAM-softmax uses scale 30 and margin 0.4.
    • For each class, 500 training utterances are sampled as a cohort. The mean cosine similarity to each cohort is normalized and averaged 50/50 with the softmax scores.
  • Training recipe.
    • Random 5-second crops are used for the first 50 epochs, then 4-second crops, for 100 epochs in total.
    • Optimization uses Adam with a triangular learning-rate schedule and a batch size of 256.
    • Augmentation comes from MUSAN noise, QMUL impulse responses, and tempo changes of 0.9/1.1, all via Kaldi.

Interesting ideas

  • I1. At decision time, the classifier's probabilities are mixed with an exemplar check: how close is the input to random samples of each class? This makes a second, instance-based opinion part of every decision.
  • I2. The model first trains on longer crops, then switches to shorter ones, so it holds up on brief inputs it will meet at test time. Matching the training length distribution to deployment is a cheap robustness trick.
  • I3. Fusing a small grid of diverse systems (two architectures × two feature types) with equal weights adds 0.8–2 points over the best single member. No tuned meta-learner is needed.
  • I4. Training labels come cheaply from metadata (video country), while the evaluation sets are human-verified. Noisy-but-large supervision is paired with a small trustworthy test.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): When choosing a VerbNet class or frame for a predicate, blend the model score with similarity to a stored cohort of labelled examples per class. The nearest cohort members can then be shown as evidence for the choice.
  • I2 (tree): Train gparse first on full sentences, then on clipped fragments and short clauses, so headlines, answers and dialogue turns parse as reliably as long prose.
  • I3 (learn): Witness parsers that differ in both architecture and input representation can be combined with plain uniform voting when building silver data. This keeps the agreement step simple and auditable.
  • I4 (learn): Silver trees or frames inferred from sources such as document metadata could be used for training, while evaluation stays on a hand-checked gold slice. That way noise in training does not contaminate the measurements.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ajinkya Kulkarni and Hanan Aldarmaki. 2023. Yet Another Model for Arabic Dialect Identification. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.37

@inproceedings{kulkarni2023yet,
  title = {{Yet Another Model for Arabic Dialect Identification}},
  author = {Ajinkya Kulkarni and Hanan Aldarmaki},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.37},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.37}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed M. Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James R. Glass, Steve Renals, and Khalid Choukri. 2019. The mgb-5 challenge: Recognition and dialect identification of dialectal arabic speech. 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1026–1033. find
  2. Ahmed M. Ali, Stephan Vogel, and Steve Renals. 2017. Speech recognition challenge in the wild: Arabic mgb-3. 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 316– 322. Recognition and Understanding Workshop (ASRU), pages 360–367.
  3. Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, and Xiangzhan Yu. 2021. Unispeech-sat: Universal speech representation learning with speaker aware pre-training. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6152– 6156. ↗
  4. Jiankang Deng, J. Guo, and Stefanos Zafeiriou. 2018. Arcface: Additive angular margin loss for deep face recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4685–4694. find
  5. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. Interspeech. find
  6. Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. ↗
  7. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdel rahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460.
  8. Sameer Khurana, Maryam Najafian, Ahmed M. Ali, Tuka Al Hanai, Yonatan Belinkov, and James R. Glass. 2017. Qmdis: Qcri-mit advanced dialect identification system. In Interspeech. find
  9. Lam Hong Lee, Chin Heng Wan, Rajprasad Rajkumar, and Dino Isa. 2012. An enhanced support vector machine classification framework by using euclidean distance function for text document categorization. Applied Intelligence, 37:80–99. ↗
  10. Liam Lonergan, Mengjie Qian, Neasa Ní Chiaráin, Christer Gobl, and Ailbhe Ní Chasaide. 2023. Towards spoken dialect identification of irish. arXiv preprint arXiv:2307.07436. ↗
  11. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, and Michael Auli. 2022. XLS-R: self-supervised cross-lingual speech representation learning at scale. In Interspeech 2022, 23rd Annual Conference of the International Speech Communication Association, Incheon, Korea, 18-22 September 2022, pages 2278–2282. ISCA.
  12. Xiaoxiao Miao and Ian Mcloughlin. 2019. Lstm-tdnn with convolutional front-end for dialect identification in the 2019 multi-genre broadcast challenge. ArXiv, abs/1912.09003. ↗
  13. Ahmet Emin Bulut, Qian Zhang, Chunlei Zhang, Fahimeh Bahmaninezhad, and John H. L. Hansen. 2017. Utd-crss submission for mgb-3 arabic dialect identification: Front-end and back-end advancements on broadcast speech. 2017 IEEE Automatic Speech Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, Jan Silovsky, Georg Stemmer, and Karel Vesely. 2011. The kaldi speech recognition toolkit.
  14. Tam T Nguyen, Kuiyu Chang, and Siu Cheung Hui. 2013. Supervised term weighting centroid-based classifiers for text categorization. Knowledge and information systems, 35:61–85. ↗
  15. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society. IEEE Catalog No.: CFP11SRW-USB. find
  16. Rajendra Kumar Roul and Kushagr Arora. 2017. A modified cosine-similarity based log kernel for support vector machines in the domain of text classification. In Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017), pages 338–347. universal performance benchmark. arXiv preprint arXiv:2105.01051. ↗
  17. Omar Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40:171–202. ↗
  18. Suwon Shon, Ahmed M. Ali, and James R. Glass. 2017. Mit-qcri arabic dialect identification system for the 2017 multi-genre broadcast challenge. 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 374–380.
  19. David Snyder, Guoguo Chen, and Daniel Povey. 2015. Musan: A music, speech, and noise corpus. arXiv preprint arXiv:1510.08484. ↗
  20. David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur. 2018. X-vectors: Robust dnn embeddings for speaker recognition. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5329–5333. find
  21. Rebecca Stewart and Mark Sandler. 2010. Database of omnidirectional and b-format room impulse responses. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 165–168. IEEE. find
  22. Peter Sullivan, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2023. On the Robustness of Arabic Speech Dialect Identification. In Proc. IN-TERSPEECH 2023, pages 5326–5330.
  23. Rong Tong, Bin Ma, Donglai Zhu, Haizhou Li, and Eng Siong Chng. 2006. Integrating acoustic, prosodic and phonotactic features for spoken language identification. In 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, volume 1, pages I–I. find
  24. Karthika Vijayan, Haizhou Li, Hanwu Sun, and Kong Aik Lee. 2018. On the importance of analytic phase of speech signals in spoken language recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5194–5198. find
  25. Shu wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdel rahman Mohamed, and Hung yi Lee. 2021. Superb: Speech processing universal performance benchmark. In Interspeech. find
  26. Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y Lin, Andy T Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, et al. 2021. Superb: Speech processing find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kulkarni-2023-yet-another-model-arabic-dialect/