Elamur Research⌕
☰
home / papers / 2019 / DARTS: Dialectal Arabic Transcription System

DARTS: Dialectal Arabic Transcription System

Summary

DARTS is a speech-to-text system for Egyptian Arabic, a dialect with very little transcribed audio and no settled spelling conventions. The authors start from a large, mostly Modern Standard Arabic broadcast corpus (MGB-2, about 1,200 hours of Al Jazeera programs). They move to the dialect by training on that corpus together with the small Egyptian set from MGB-3, about 16 hours of YouTube speech. They also collected over 500 hours of unlabeled Egyptian YouTube audio and used it through semi-supervised lattice-free MMI training.

The acoustic model is a hybrid HMM-DNN. It stacks a 2D CNN front end, TDNN layers and interleaved LSTM/TDNN blocks, and it is trained with LF-MMI plus a cross-entropy branch. Units are graphemes rather than phonemes. Decoding uses n-gram language models, and N-best lists are rescored with an RNN language model. Combining several systems gave the lowest reported word error rate on MGB-3.

The work matters because it shows that cheap, simple steps can make up for scarce dialect transcriptions. Pooling corpora, adding unlabeled in-domain audio and avoiding hand-built pronunciation lexicons were enough.

Key points

  • Data:
    • MGB-2 provides about 1,200 hours of broadcast speech from 19 program series (2005–2015).
    • MGB-3 provides 16 hours of mostly Egyptian YouTube speech. Its adaptation and development splits were merged for training.
    • Another 500+ hours of unlabeled Egyptian YouTube audio came from hand-picked channels, searched by genre keywords and checked with a five-way Arabic dialect classifier.
  • Language-model text: a background corpus of over 130 million words crawled from AlJazeera.net (2000–2011), transcribed in Buckwalter format.
  • MGB-2 segment selection: lightly supervised decoding produced alignments. Segments were kept according to thresholds on Matched Error Rate, Phone Matched Error Rate and average word duration.
  • Acoustic training: Multi-LF-MMI, meaning LF-MMI combined with a cross-entropy branch for stability in deep networks, implemented in Kaldi. The semi-supervised variant replaces the reference transcription in the numerator with a sum over hypotheses produced by a seed model.
  • Grapheme lexicon: chosen because it requires no linguistic expertise. The authors argue its label noise may act as a regularizer against overfitting to one dialect.

Interesting ideas

  • I1. When references are loose paraphrases, decode the data lightly, measure how far the output drifts from the reference, and keep only segments within a threshold. Unreliable labels become a usable training subset. — For Mova (learn): Silver UD trees could be admitted only when a parser's output stays close to a rough external reference, such as an LLM-teacher parse, with a tunable mismatch cutoff.
  • I2. For unlabeled data, the training target is a weighted set of competing hypotheses from a seed model rather than a single guessed transcript. This keeps the model from overcommitting to one possibly wrong guess. — For Mova (learn): Training on unlabeled text could use the n-best parses agreed by the witness parsers, weighted by score, instead of forcing one 1-best silver tree.
  • I3. A whole-sequence training objective was unstable on its own in deep networks, so it was paired with a simpler per-frame loss in a multi-task setup. The local loss steadies the global one. — For Mova (tree): gparse could pair a per-token head/label cross-entropy with a whole-tree structured loss, letting the token loss stabilize the structured objective.
  • I4. Crawled audio was screened with a dialect identifier before use, so the extra data actually matched the target variety. — For Mova (learn): Web text could pass through a register or dialect classifier before silver parsing, keeping domain-mismatched sentences out of the training pool.
  • I5. The system models raw written symbols instead of curated linguistic units, trading a little noise for independence from expert resources. Given enough data, the paper finds this costs little. — For Mova (tokens): A character-level fallback for out-of-vocabulary morphology and lemmas could cover rare or nonstandard spellings without hand-written rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

2 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1909.12163 · arXiv 1909.12163

@misc{khurana2019darts,
  title = {{DARTS: Dialectal Arabic Transcription System}},
  author = {Khurana, Sameer and Ali, Ahmed and Glass, James},
  year = {2019},
  doi = {10.48550/arxiv.1909.12163},
  eprint = {1909.12163},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1909.12163}
}

References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Ali and Steve Renals, “Word error rate estimation for speech recognition: e-wer,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018, pp. 20–24. find
  2. Ahmed Ali, Multi-dialect Arabic broadcast speech recognition, Ph.D. thesis, The University of Edinburgh, 2018. find
  3. Peter Bell, Mark JF Gales, Thomas Hain, Jonathan Kilgour, Pierre Lanchantin, Xunying Liu, Andrew McParland, Steve Renals, Oscar Saz, Mirjam Wester, et al., “The mgb challenge: Evaluating multi-genre broadcast media recognition,” in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE, 2015, pp. 687–693. find
  4. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang, “The mgb-2 challenge: Arabic multi-dialect broadcast media recognition,” in 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2016, pp. 279–284. find our card
  5. Ahmed Ali, Stephan Vogel, and Steve Renals, “Speech recognition challenge in the wild: Arabic mgb-3,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2017, pp. 316–322. find our card
  6. Sameer Khurana and Ahmed Ali, “QCRI advanced transcription system (qats) for the arabic multi-dialect broadcast media recognition: Mgb-2 challenge,” in 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2016, pp. 292–298. find
  7. Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur, “Audio augmentation for speech recognition,” in Sixteenth Annual Conference of the International Speech Communication Association, 2015. find
  8. Vimal Manohar, Hossein Hadian, Daniel Povey, and Sanjeev Khudanpur, “Semi-supervised training of acoustic models using lattice-free mmi,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4844–4848. find
  9. Suwon Shon, Ahmed Ali, and James Glass, “Convolutional neural network and language embeddings for end-to-end dialect recognition,” in Proc. Odyssey 2018 The Speaker and Language Recognition Workshop, 2018, pp. 98–104. find
  10. Yu Wang, Xie Chen, Mark JF Gales, Anton Ragni, and Jeremy HM Wong, “Phonetic and graphemic systems for multi-genre broadcast transcription,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018. find
  11. Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al., “Deep speech 2: End-to-End speech recognition in English and Mandarin,” in International conference on machine learning, 2016, pp. 173–182. find our card
  12. Suyoun Kim, Takaaki Hori, and Shinji Watanabe, “Joint ctc-attention based end-to-end speech recognition using multi-task learning,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 4835–4839. find
  13. Yajie Miao, Mohammad Gowayyed, and Florian Metze, “Eesen: End-to-end speech recognition using deep rnn models and wfst-based decoding,” in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE, 2015, pp. 167–174. find
  14. Qiantong Xu Jeff Cai Jacob Kahn Gabriel Synnaeve Vitaliy Liptchinsky Ronan Collobert Vineel Pratap, Awni Hannun, “wav2letter++: The fastest open-source speech recognition system,” CoRR, vol. abs/1812.07625, 2018. ↗
  15. Pawel Swietojanski, Arnab Ghoshal, and Steve Renals, “Revisiting hybrid and gmm-hmm system combination techniques,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2013, pp. 6744–6748. find
  16. Sigurdur Sigurdsson, Kaare Brandt Petersen, and Tue Lehn-Schiøler, “Mel frequency cepstral coefficients: An evaluation of robustness of mp3 encoded music,” in IS-MIR, 2006, pp. 286–289. find
  17. Karel Veselỳ, Arnab Ghoshal, Lukás Burget, and Daniel Povey, “Sequence-discriminative training of deep neural networks,” in Interspeech, 2013. find
  18. Daniel Povey, Vijayaditya Peddinti, Daniel Galvez, Pegah Ghahremani, Vimal Manohar, Xingyu Na, Yiming Wang, and Sanjeev Khudanpur, “Purely sequence-trained neural networks for asr based on lattice-free mmi.,” in Interspeech, 2016. find
  19. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al., “The kaldi speech recognition toolkit,” Tech. Rep., IEEE Signal Processing Society, 2011. find
  20. Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur, “A time delay neural network architecture for efficient modeling of long temporal contexts,” in Sixteenth Annual Conference of the International Speech Communication Association, 2015. find
  21. Yann LeCun, Yoshua Bengio, et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, pp. 1995, 1995. find
  22. Sepp Hochreiter and Jürgen Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997. find
  23. Ruhi Sarikaya and John HL Hansen, “High resolution speech feature parametrization for monophone-based stressed speech recognition,” IEEE signal processing letters, vol. 7, no. 7, pp. 182–185, 2000. find
  24. Tomáš Mikolov, Statistical language models based on neural networks, Ph.D. thesis, BRNO UNIVERSITY OF TECHNOLOGY, 2012. find
  25. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. ↗
  26. Peter Smit, Siva Reddy Gangireddy, Seppo Enarvi, Sami Virpioja, and Mikko Kurimo, “Aalto system for the 2017 arabic multi-genre broadcast challenge,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2017, pp. 338–345. find
  27. Suwon Shon and Ahmed Ali, “Mgb-5: Arabic dialect identification across 17 dialects and moroccan speech recognition,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2019. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khurana-2019-darts-dialectal-arabic-transcription-system/