Elamur Research⌕
☰
home / papers / 2026 / Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language Therapy

Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language Therapy

Summary

Arabic-speaking children with autism often get too little speech and language therapy. Qualified Speech-Language Pathologists (SLPs) are scarce and concentrated in cities, and most digital therapy material is built for English speakers, with objects, scenes and routines that feel foreign to these children. The authors built Digital Harf, a web platform co-designed with SLPs from several Middle Eastern centers. It brings clinician-led therapy into the home through three modules: language therapy, speech intelligibility and picture description. The modules adapt to each child's progress, and a caregiver assistant supports parents. To solve the shortage of content, they built the Agentic Synthetic Data Engine (ASDE). It produces images, prompts and language tasks under explicit clinical and cultural guidelines and screens them with automatic judges. In an expert study, 13 licensed SLPs accepted 90.1% of ASDE output that had not been curated or filtered, and they rated the platform's cultural and linguistic fit highly. The paper also gives a replication blueprint for other low-resource languages, in which localization is done through content and configuration files rather than code. A clinical trial measuring outcomes in children is in progress.

Key points

  • Three therapy modules (language therapy, speech intelligibility, picture description) share one personalized workflow, with adaptive image selection and an AI caregiver assistant that uses tool-calling.
  • ASDE content reached a 90.1% clinical acceptance rate from 13 SLPs at three Saudi centers, with no manual editing or selection.
  • The sample question bank covers 12 subdomains, including objects, actions, colors, opposites, negatives, prepositions, parts/whole, yes/no and WH questions, with only a few exemplars each.
  • Questions are generated in English and then translated into Arabic by an LLM that preserves clinical meaning. The authors note that generating directly in Arabic is also supported.
  • The recommended rollout has four steps: write a cultural guideline document and a seed question bank, edit the configuration files, generate in batches of 10–20 for clinician review, and finally localize the assistant and UI.

Interesting ideas

  • I1. A small set of expert-written example questions for each subdomain improves output quality more than increasing the generation budget does. — For Mova (learn): Prompts for the LLM teacher tiers could include a few hand-picked gold examples for each hard construction type (coordination, relative clauses, ellipsis), instead of relying on many more sampled silver items.
  • I2. All domain and cultural knowledge sits in one human-editable configuration file (topic lists, style templates, answer strategies), so experts can adapt the system without touching code. — For Mova (world): Measures, value scales and verb-class lists could live in declarative files that linguists edit directly, keeping the world-building logic unchanged and every knowledge source inspectable.
  • I3. Automatic multi-judge filtering is combined with early human review of small batches. The human review is meant to test whether the written guidelines are specific enough, not to catch individual errors. — For Mova (learn): Alongside agreement between witness parsers, a person could audit a sample of 10–20 sentences per new rule or treebank batch to show where the agreement criteria are too loose.
  • I4. A variation layer rotates age bands, settings and topic keywords so that generated items stay diverse rather than repeating one template. — For Mova (answer): Synthetic maths word problems and school-test items could rotate settings, quantities and actors in the same way, which tests reader robustness instead of rewarding memorized surface patterns.
  • I5. Quality is reported as an acceptance rate on uncurated output, which gives an honest measure with no cherry-picking. — For Mova (back): Generation and text → graph → text compression could be scored on random unselected outputs judged by people, so the reported quality reflects real behavior.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

1 paper of our library cites it (of the 140884 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2607.27212 · arXiv 2607.27212

@misc{azad2026digital,
  title = {{Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language Therapy}},
  author = {Azad, Asif and Hossain, Mohammad Sadat and Shanto, MD Sadik Hossain and Boughorbel, Sabri and Aljouie, Abdulrhman and Alwuqaysi, Bdour and Bokhari, Yahya and Sindi, Ayah Othman and Hoque, Ehsan},
  year = {2026},
  doi = {10.48550/arxiv.2607.27212},
  eprint = {2607.27212},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2607.27212}
}

References

31 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Haneen Almurashi, Rahma Bouaziz, Wallaa Alharthi, Mohammed Al-Sarem, Mohammed Hadwan, and Slim Kammoun. 2022. Augmented Reality, Serious Games and Picture Exchange Communication System for People with ASD: Systematic Literature Review and Future Directions. Sensors 22, 3 (2022), 1250. doi:10.3390/s22031250 ↗
  2. Ahmed Adel Attia, Jing Liu, Wei Ai, Dorottya Demszky, and Carol Espy-Wilson. 2024. Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 74–80. doi:10.1609/aies.v7i1.31618 ↗
  3. Harini Atturu, Somasekhar Naraganti, and Bugatha Rajvir Rao. 2025. Effectiveness of Artificial Intelligence–Based Platform in Administering Therapies for Children With Autism Spectrum Disorder: 12-Month Observational Study. JMIR Neurotechnology 4 (2025), e70589. doi:10.2196/70589 ↗
  4. Geertruida Aline Attwell, Kwabena Ebo Bennin, and Bedir Tekinerdogan. 2022. A Systematic Review of Online Speech Therapy Systems for Intervention in Childhood Speech Communication Disorders. Sensors 22, 24 (2022), 9713. doi:10.3390/s22249713 ↗
  5. Asif Azad, MD Shanto, Mohammad Sadat Hossain, Bdour Alwuqaysi, Sabri Boughorbel, Yahya Bokhari, Abdulrhman Aljouie, Ayah Othman Sindi, and Ehsan Hoque. 2026. Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment. arXiv preprint arXiv:2604.06191 (2026). ↗ our card
  6. Dasom Choi, Sunok Lee, Sung-In Kim, Kyungah Lee, Hee Jeong Yoo, Sangsu Lee, and Hwajung Hong. 2024. Unlock Life with a Chat(GPT): Integrating Conversational AI with Large Language Models into Everyday Lives of Autistic Individuals. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 72, 17 pages. doi:10.1145/3613904.3641989 ↗
  7. Dasom Choi, SoHyun Park, Kyungah Lee, Hwajung Hong, and Young-Ho Kim. 2025. AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–25. doi:10.1145/3706598.3713792 ↗
  8. C. H. Chueh, T. H. Chiang, P. W. Pan, K. L. Lin, Y. S. Lu, S. H. Tuan, C. R. Lin, I. C. Huang, and H. S. Cheng. 2025. Implementation of a Generative AI-Powered Digital Interactive Platform for Clinical Language Therapy in Children with Language Delay: A Pilot Study. Life 15, 10 (October 2025), 1628. doi:10.3390/life15101628 ↗
  9. Aayushi Dangol, Aaleyah Lewis, Hyewon Suh, Xuesi Hong, Hedda Meadan, James Fogarty, and Julie A. Kientz. 2025. “I Want to Think Like an SLP”: A Design Exploration of AI-Supported Home Practice in Speech Therapy. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–22. doi:10.1145/3706598.3713986 ↗
  10. Chinmoy Deka, Abhishek Shrivastava, Ajish K. Abraham, Saurabh Nautiyal, and Praveen Chauhan. 2025. AI-based automated speech therapy tools for persons with speech sound disorder: a systematic literature review. Speech, Language and Hearing 28, 1 (2025), 2359274. doi:10.1080/2050571X. 2024.2359274 ↗
  11. Chinmoy Deka, Abhishek Shrivastava, and Rishav Kumar. 2025. Towards human-centered AI in speech therapy: perspectives from a low-resource setting. Universal Access in the Information Society 25, 1 (December 2025), 16. doi:10.1007/s10209-025-01288-2 ↗
  12. S. Ghali, F. Abdalla, and A. Aseeri. 2023. Opportunities and Challenges for Audiology and Speech-Language Pathology Services in Arabic-Speaking Middle East Countries. Perspectives of the ASHA Special Interest Groups 8, 1 (2023), 217–225. doi:10.1044/2022_persp-22-00103 ↗
  13. Adam Hair, Kirrie J. Ballard, Constantina Markoulli, Jacqueline Mckechnie, Penelope Monroe, Beena Ahmed, and Ricardo Gutierrez-Osuna. 2021. A Longitudinal Evaluation of Tablet-Based Child Speech Therapy with Apraxia World. ACM Transactions on Accessible Computing 14, 1 (2021), 1–26. doi:10.1145/3433607 ↗
  14. H. Khamees, A. Gulati, A. Vatsa, M. Aldosari, and E. Hoque. 2025. Estimating autism prevalence among children in the Kingdom of Saudi Arabia: A comprehensive multisource benchmarking approach. Saudi Medical Journal 46, 11 (November 2025), 1276–1279. doi:10.15537/smj.2025.46.11.20250781 ↗
  15. KidzCare Pediatrics. 2025. KidzCare App. https://kidzcare.app/en/ Accessed: 2026-05-01. ↗
  16. Do Hyung Kim, Joo Won Jeong, Dayoung Kang, Taekyung Ahn, Yeonjung Hong, Younggon Im, Jaewon Kim, Min Jung Kim, and Dae-Hyun Jang. 2025. Usefulness of Automatic Speech Recognition Assessment of Children With Speech Sound Disorders: Validation Study. Journal of Medical Internet Research 27 (2025), e60520. doi:10.2196/60520 ↗
  17. King Salman Center for Disability Research. 2024. Disability Statistics in the Kingdom of Saudi Arabia. https://www.kscdr.org.sa/en/stats. Accessed: September 11, 2024. ↗
  18. Daniel Koronthály, Adrian Heffelman, Hamza Magsi, Wilson Zuber, Moushumi Sharmin, and Shameem Ahmed. 2025. The Co-Evolution of Language, Technology, and Culture in Autism Computing Research: Lessons Learned From ACM Digital Library. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 9, 4 (2025), 1–31. doi:10.1145/3770638 ↗
  19. Shuzhong Lai, Chenxi Li, Junhong Lai, Yucun Zhong, Chenyu Yan, Xiang Li, Haifeng Li, Gang Pan, Lin Yao, and Yueming Wang. 2025. ASD-iLLM:An Intervention Large Language Model for Autistic Children based on Real Clinical Dialogue Intervention Dataset. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 8058–8079. doi:10.18653/v1/2025.findings-emnlp.427 ↗
  20. Seonwoo Lee, Jihyun Mun, Sunhee Kim, HyunJu Park, Suvin Yang, HyunDon Kim, SeungJae Noh, WonBin Kim, and Minhwa Chung. 2024. Automatic Speech Recognition and Assessment Systems Incorporated into Digital Therapeutics for Children with Autism Spectrum Disorder. 328–335. doi:10.1007/978-3-031-62849-8_40 ↗
  21. Aaleyah Lewis, Aayushi Dangol, Hyewon Suh, Abbie Olszewski, James Fogarty, and Julie A. Kientz. 2025. Exploring AI-Based Support in Speech-Language Pathology for Culturally and Linguistically Diverse Children. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–19. doi:10.1145/3706598.3714131 ↗
  22. Yue Lyu, Pengcheng An, Yage Xiao, Zibo Zhang, Huan Zhang, Keiko Katsuragawa, and Jian Zhao. 2023. Eggly: Designing Mobile Augmented Reality Neurofeedback Training Games for Children with Autism Spectrum Disorder. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 2, Article 67 (June 2023), 29 pages. doi:10.1145/3596251 ↗
  23. Yue Lyu, Di Liu, Pengcheng An, Xin Tong, Huan Zhang, Keiko Katsuragawa, and Jian Zhao. 2024. EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-Infused and AR Activities. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 4 (2024), 1–36. doi:10.1145/3699738 ↗
  24. Gabriela Marcu, Anind K. Dey, and Sara Kiesler. 2012. Parent-Driven Use of Wearable Cameras for Autism Support: A Field Study with Families. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. Association for Computing Machinery, New York, NY, USA, 401–410. doi:10.1145/2370216.2370277 ↗
  25. Microsoft. [n. d.]. Use pronunciation assessment. https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-pronunciation-assessment?pivots=programming-language-python Microsoft Learn documentation. Accessed: May 2, 2026. ↗
  26. Khadijeh Moulaei, Fatemeh Dinari, Mobina Hosseini, Sohrab Almasi, Babak Sabet, Romina Anabestani, and Mohammad Reza Afrash. 2025. Exploring tele-speech therapy: A scoping review of interventions, applications, benefits, and challenges. International Journal of Medical Informatics 195 (2025), 105784. doi:10.1016/j.ijmedinf.2025.105784 ↗
  27. Nina Perry, Calum Sun, Mark Munro, K. A. Boulton, and Adam J. Guastella. 2024. AI technology to support adaptive functioning in neurodevelopmental conditions in everyday environments: a systematic review. npj Digital Medicine 7 (2024), 362. doi:10.1038/s41746-024-01355-7 ↗
  28. Kaśka Porayska-Pomsta, Alyssa M. Alcorn, Katerina Avramides, Sandra Beale, Sara Bernardini, Mary Ellen Foster, Christopher Frauenberger, Judith Good, Karen Guldberg, Wendy Keay-Bright, Lila Kossyvaki, Oliver Lemon, Marilena Mademtzi, Rachel Menzies, Helen Pain, Gnanathusharan Rajendran, Annalu Waller, Sam Wass, and Tim J. Smith. 2018. Blending Human and Artificial Intelligence to Support Autistic Children’s Social Communication Skills. ACM Transactions on Computer-Human Interaction 25, 6 (2018), 1–35. doi:10.1145/3271484 ↗
  29. Hyewon Suh, Aayushi Dangol, Hedda Meadan, Carol A. Miller, and Julie A. Kientz. 2024. Opportunities and Challenges for AI-Based Support for Speech-Language Pathologists. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work. Association for Computing Machinery, New York, NY, USA, 1–14. doi:10.1145/3663384.3663387 ↗
  30. Michela M. Tinelli, Aine Roddy, Martin Knapp, et al. 2023. Economic evaluation of anti-epileptic medicines for autistic children with epilepsy. Journal of Autism and Developmental Disorders (2023). doi:10.1007/s10803-023-05941-8 ↗
  31. Matej Zajc, Andreja Istenič Starčič, Maja Lebeničnik, and Mateja Gačnik. 2018. Tablet game-supported speech therapy embedded in children’s popular practices. Behaviour & Information Technology 37, 7 (2018), 693–702. doi:10.1080/0144929X.2018.1474253 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/azad-2026-digital-harf-clinically-integrated-multimodal/