Elamur Research⌕
☰
home / papers / 2022 / AI-based Arabic Language and Speech Tutor

AI-based Arabic Language and Speech Tutor

Summary

The authors develop AI-ALST, an intelligent tutor for students of Moroccan Arabic (Darija) at the University of Arizona. Machine-learning pronunciation training had not previously been applied to this dialect. Dialectal Arabic is low-resource and has few qualified human tutors, so the system aims to give learners an always-available self-study space that goes past multiple-choice drills. Its core is a computer-assisted pronunciation training (CAPT) component built on four parts:

  • MFCC acoustic features,
  • a bidirectional LSTM,
  • an attention layer,
  • cost-sensitive learning to handle the scarcity of mispronounced examples.

The material comes from two multimedia Moroccan Arabic eTextbooks used in blended, flipped-classroom courses. The first evaluation is narrow: word pronunciation from lesson 1 only. Scoring uses accuracy, precision, recall and F1-score. The authors report that the model reliably identifies pronunciation errors. The work is a proof of concept for AI tutoring in an under-served Arabic dialect. Planned next steps are adaptive exercises, explainable error diagnoses and study of first-language accent effects.

Key points

  • The CAPT pipeline is MFCC features → BiLSTM → attention mechanism, with a cost-based strategy for class imbalance. It was evaluated on lesson 1 word pronunciation.
  • The paper frames tutoring systems as three parts: domain knowledge, a student model that keeps learner history and drives adaptation, and a teaching module.
  • The course material is organized as follows:
    • Level I has seven modules of three lessons each.
    • Each lesson splits into vocabulary/structures, listening comprehension and speaking assessment.
    • The modules come from a needs survey of Arabic Language Flagship Program stakeholders.
  • The related-work section cites the Arabic Learner Corpus (ALC): 282,732 words and 1,585 materials from 942 students of 67 nationalities. It also lists Arabic dialect chatbots BOTTA (Egyptian) and Nabiha (Saudi).
  • Chatbot knowledge bases are grouped into rule-based, retrieval-based and generative-based types.

Interesting ideas

  • I1. Useful tutoring feedback should go beyond flagging an error: it should name the mistake, trace where it originates, and propose targeted follow-up practice. — For Mova (answer): When a student-reader fails a school test item, report which pipeline stage caused the miss (parse, frame, world state) and pick follow-up items that exercise that stage.
  • I2. Correct examples vastly outnumber errors, so the authors make misclassifying the rare class more expensive instead of rebalancing the data. — For Mova (tree): Cost-weighting could raise accuracy on rare UD relations (e.g. vocative, dislocated) in the perceptron parser or gparse without resampling treebanks.
  • I3. Imbalanced detection is scored with precision, recall and F1 alongside accuracy, since accuracy alone hides poor performance on the rare class. — For Mova (learn): Silver-data and repair-rule evaluations should always report per-class precision/recall for rare phenomena, so a high overall score cannot mask a failing minority case.
  • I4. A persistent student model records each learner's history and lets the system change what it presents next. — For Mova (answer): Mova's readers could keep a history of failed question types across test suites, so training and self-play focus on the weakest skills.
  • I5. The authors plan to give explainable diagnoses for each detected pronunciation error, rather than leave the classifier as an opaque judge. — For Mova (learn): This supports Mova's rule that every decision, including learned ones, must carry a human-readable justification that can be audited.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2210.12346 · arXiv 2210.12346

@misc{shao2022ai,
  title = {{AI-based Arabic Language and Speech Tutor}},
  author = {Shao, Sicong and Alharir, Saleem and Hariri, Salim and Satam, Pratik and Shiri, Sonia and Mbarki, Abdessamad},
  year = {2022},
  doi = {10.48550/arxiv.2210.12346},
  eprint = {2210.12346},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2210.12346}
}

References

27 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y.-F. Wang and S. Petrina, “Using learning analytics to understand the design of an intelligent language tutor–Chatbot lucy,” Editorial Preface 4, no. 11, 124-131, 2013. find
  2. G. Richards, “Book review: Technology-mediated Learning Environments for Young English Learners (L. Leanne Parker),” Educational Technology & Society, 12 (2), 334-336, 2009. find
  3. A. Kerlyl, P. Hall, and S. Bull, “Bringing chatbots into education: Towards natural language negotiation of open learner models,” in International conference on innovative techniques and applications of artificial intelligence, pp. 179-192. Springer, London, 2006. find
  4. E. Wenger. Artificial intelligence and tutoring systems: computational and cognitive approaches to the communication of knowledge. Morgan Kaufmann, 2014. find
  5. J. E. Greer and Gordon I. McCalla, “Student modelling: the key to individualized knowledge-based instruction,” Vol. 125. Springer Science & Business Media, 2013. find
  6. J. Fouz-González, “Trends and directions in computer-assisted pronunciation training,” Investigating English Pronunciation, 314-342, 2015. find
  7. A. Diment, E. Fagerlund, A. Benfield, and T. Virtanen, “Detection of typical pronunciation errors in non-native English speech using convolutional recurrent neural networks,” in International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2019. find
  8. M. A. Mansour, Mohamed Abdelmageed, “The absence of Arabic corpus linguistics: a call for creating an Arabic national corpus,” International Journal of Humanities and Social Science 3, no. 12, 81-90, 2013. find
  9. M. Zaki, “Corpus based teaching in the Arabic classroom: Theoretical and practical perspectives,” International Journal of Applied Linguistics 27, no. 2, 514-541, 2017. find
  10. J. Weizenbaum, “ELIZA—a computer program for the study of natural language communication between man and machine,” Communications of the ACM 9, no. 1, 36-45, 1966. find
  11. J. L. Hutchens and M. D. Alder, “Introducing megahal,” in New Methods in Language Processing and Computational Natural Language Learning, 1998. find
  12. L. Fryer and R. Carpenter, “Bots as language learning tools,” Language Learning & Technology 10, no. 3, 8-14, 2006. find
  13. A. Ahmed, N. Ali, M. Alzubaidi, W. Zaghouani, A. Abd-alrazaq, and M. Househ, “Arabic Chatbot Technologies: A Scoping Review,” Computer Methods and Programs in Biomedicine Update, 2022. find
  14. S. Nakagawa, L. Wang, and S. Ohtsuka, “Speaker identification and verification by combining MFCC and phase information,” IEEE transactions on audio, speech, and language processing 20, no. 4, 2011. find
  15. A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network,” Physica D: Nonlinear Phenomena, 2020. find
  16. Y. Yu, X. Si, C. Hu, and J. Zhang, “A review of recurrent neural networks: LSTM cells and network architectures,” Neural Computation, Vol. 31, no. 7, 1235-1270, 2019. find
  17. S. Siami-Namini, N. Tavakoli, and A. S. Namin, “The performance of LSTM and BiLSTM in forecasting time series,” in IEEE International Conference on Big Data (Big Data), pp. 3285-3292. IEEE, 2019. find
  18. D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in International Conference on Learning Representations, 2014. find our card
  19. Z.-H. Zhou and X.-Y. Liu, “Training cost-sensitive neural networks with methods addressing the class imbalance problem,” IEEE Transactions on knowledge and data engineering 18, no. 1, 63-77, 2005. find
  20. J. Bang, J. Lee, G. G. Lee, and M. Chung, “Pronunciation variants prediction method to detect mispronunciations by Korean learners of English,” ACM Transactions on Asian Language Information Processing (TALIP) 13, no. 4, 1-21, 2014. find
  21. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations, 2015. find
  22. I. Hamed, D. Pavel, C. Li, M. Elmahdy, S. Abdennadher, and N. T. Vu, “Investigations on speech recognition systems for low-resource dialectal Arabic–English code-switching speech,” Computer Speech & Language, 2022. find
  23. S. Shiri, “Intercultural communicative competence development during and after language study abroad: Insights from Arabic,” Foreign Language Annals 48, no. 4, 2015. find
  24. S. Shao, C. Tunc, A. Al-Shawi, and S. Hariri, “An ensemble of ensembles approach to author attribution for internet relay chat forensics,” ACM Transactions on Management Information Systems (TMIS) 11, no. 4, 1-25, 2022. find
  25. S. Shao, C. Tunc, A. Al-Shawi, and S. Hariri, “One-class classification with deep autoencoder neural networks for author verification in internet relay chat,” in IEEE/ACS International Conference on Computer Systems and Applications, 2019. find
  26. D. Al-Ghadhban and Nora Al-Twairesh, “Nabiha: an Arabic dialect chatbot,” International Journal of Advanced Computer Science and Applications 11, no. 3, 2020. find
  27. D. A. Ali and N. Habash, “Botta: An arabic dialect chatbot,” in International Conference on Computational Linguistics: System Demonstrations, 2016. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shao-2022-ai-based-arabic-language-speech/