Elamur Research⌕
☰
home / papers / 2019 / An Arabic Multi-Domain Spoken Language Understanding System

An Arabic Multi-Domain Spoken Language Understanding System

Paper · 2019 · Mohamed Lichouri, Mourad Abbas, Rachida Djeradi et al.

Summary

The paper investigates whether one Arabic Spoken Language Understanding (SLU) setup can be carried across several dialogue domains with little expert effort. It covers both database-backed retrieval and information-extraction settings. The authors treat understanding as intent identification, which they implement as text categorization. They test it on four domains: University Schooling Management, Medical Diagnostic, Consultation, and Question-Answering based on the DAWQAS corpus. Requests are stripped of punctuation and represented with word n-grams or character features, with Arabic stop words either kept or removed. The features are vectorized with TF-IDF for the statistical classifiers and with one-hot encoding for a CNN. Seven scikit-learn classifiers and one Keras CNN are compared on a 70/30 split of each corpus.

Results differ widely by domain. Best F1 is 100% on the tiny School set, 54% on Medical, 74% on Consultation and 63% on DAWQAS. Averages across configurations are much lower. No feature combination or classifier wins consistently. The work matters mainly as an early Arabic multi-domain SLU baseline with partly released data, although each domain is in practice trained and tested on its own.

Key points

  • Corpora:
    • School: about 127 student requests, gathered in French from roughly 300 students and translated to Arabic by hand; 3 classes.
    • Medical: Doctissimo forum posts covering 7 diseases.
    • Consultation: Islamtoday website, 4 consulting types.
    • DAWQAS: 13 topical classes.
    • School and Medical are published on GitHub (licvol/Arabic-Spoken-Language-Understanding).
  • Models: MNB, BNB, LSVC, Logistic Regression, SGD, Passive Aggressive, Perceptron and a CNN. The CNN was run with 10 and 100 iterations.
  • Feature grid: 8 numbered combinations, crossing stop words kept or removed with word unigram, bigram or trigram, or a character analyzer.
  • Best F1 per domain:
    • School: 100% (Perceptron)
    • Medical: 54% (SGD)
    • Consultation: 74% (SGD and LogReg)
    • DAWQAS: 63% (LSVC and Passive Aggressive)
  • Average F1 per domain: roughly 63%, 25%, 39% and 32%, a large gap from the peak scores.
  • Conclusions drawn by the authors:
    • Whether stop words help depends on the task.
    • No classifier ports well across domains.
    • Corpus size alone does not predict the score. Note that the paper's claim that the Consultation corpus is smaller conflicts with its own Table 2.

Interesting ideas

  • I1. Each preprocessing-plus-feature setting gets a short numeric ID, so every classifier can be scored on the same small grid and the interaction between settings becomes visible.
  • I2. Discarding Arabic stop words helped some classifiers and hurt others, suggesting that function words can carry the intent of a request.
  • I3. One intent-recognition front end serves both database-backed and free-text domains, and the recognized category decides which kind of backend handles the request.
  • I4. The paper reports both the best and the average score across configurations, which shows how far the peak result sits from typical behaviour.
  • I5. Character-level analyzers were compared directly with word n-grams, and for some classifiers characters gave the best result.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): A fixed, numbered ablation grid (for example tokenizer variant × parser × repair rules on or off) would make evaluation runs comparable over time and expose fragile gains.
  • I2 (frames): This supports keeping function words such as modals, question words and negation as first-class signals when assigning speech acts, rather than filtering them early.
  • I3 (answer): A similar explicit routing step could choose between a structured lookup in the world model and a reading over the text, with the routing decision itself logged as an explanation.
  • I4 (learn): Reporting mean and spread over configurations next to the best run, alongside negative controls, would guard against celebrating lucky settings.
  • I5 (tokens): Character-level evidence could serve as an explainable fallback for unknown or misspelled words in the tagger and lemmatizer, compared against word-level features under the same evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksSpeech & spoken language · 480 works

Cite

Cite as Mohamed Lichouri, Mourad Abbas, Rachida Djeradi and Amar Djeradi. 2019. An Arabic Multi-Domain Spoken Language Understanding System. Proceedings of the 3rd International Conference on Natural Language and Speech Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lichouri2019arabic,
  title = {{An Arabic Multi-Domain Spoken Language Understanding System}},
  author = {Mohamed Lichouri, Mourad Abbas, Rachida Djeradi and Amar Djeradi},
  booktitle = {Proceedings of the 3rd International Conference on Natural Language and Speech Processing},
  year = {2019},
  url = {https://aclanthology.org/W19-7407/}
}

References

21 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Drs Claude Malhuret Laurent Alexandre. 2000. Sant et bien łtre avec doctissimo. http://www.doctissimo.fr/. [Online; accessed 07/08/2018]. ↗
  2. C. Chelba and A. Acero. 2006. Adaptation of maximum entropy capitalizer: Little data can help a lot. Computer Speech & Language, 20(4):382–399. ↗
  3. H. Daumé-III. 2007. Frustratingly Easy Domain Adaptation. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL), Columbus, Ohio, USA.
  4. Hal Daumé-III and Jagadeesh Jagarlamudi. 2011. Domain adaptation for machine translation by mining unseen words. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, pages 407–412.
  5. Hal Daumé-III and Daniel Marcu. 2006. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26. ↗
  6. Walaa Saber Ismail and Masun Nabhan Homsi. 2018. Dawqas: A dataset for arabic why question answering system. Procedia computer science, 142:123– 131. find
  7. A Jaech, L Heck, and M Ostendorf. 2016. Domain adaptation of recurrent neural networks for natural language understanding. In http://arxiv.org/abs/1604.00117. ↗
  8. Ridong Jiang, Rafael E Banchs, Seokhwan Kim, Kheng Hui Yeo, Arthur Niswar, and Haizhou Li. 2014. Web-based multimodal multi-domain spoken dialogue system. In Proceedings of 5th International Workshop on Spoken Dialog Systems. find
  9. Sangkeun Jung, Cheongjae Lee, Kyungduk Kim, Minwoo Jeong, and Gary Geunbae Lee. 2009. Data-driven user simulation for automated evaluation of spoken dialog systems. Computer Speech & Language, 23(4):479–509. ↗
  10. Y Kim and R Sarikaya. 2015. New Transfer Learning Techniques For Disparate Label Sets. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Beijing, China. find
  11. Kazunori Komatani, Satoshi Ikeda, Tetsuya Ogata, and Hiroshi G Okuno. 2008. Managing out-of-grammar utterances by topic estimation with domain extensibility in multi-domain spoken dialogue systems. Speech Communication, 50(10):863–870. ↗
  12. Cheongjae Lee, Sangkeun Jung, Seokhwan Kim, and Gary Geunbae Lee. 2009. Example-based dialog modeling for practical multi-domain dialog system. Speech Communication, 51(5):466–484. ↗
  13. Fabrice Lefevre, Djamel Mostefa, Laurent Besacier, Yannick Esteve, Matthieu Quignard, Nathalie Camelin, Benoit Favre, Bassam Jabaian, and Lina Maria Rojas Barahona. 2012. Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the portmedia corpora. In The International Conference on Language Resources and Evaluation. find
  14. Mohamed Lichouri, Mourad Abbas, Abed Alhakim Freihat, and Dhiya El Hak Megtouf. 2018a. Word-level vs sentence-level language identification: Application to algerian and arabic dialects. Procedia Computer Science, 142:246–253. find
  15. Mohamed Lichouri, Amar Djeradi, and Rachida Djeradi. 2015. A new automatic approach for understanding the spontaneous utterance in human-machine dialogue based on automatic text categorization. In Proceedings of the International Conference on Intelligent Information Processing, Security and Advanced Communication, page 50. ACM. find
  16. Mohamed Lichouri, Rachida Djeradi, and Amar Djeradi. 2018b. Combining topic-based model and text categorisation approach for utterance understanding in human-machine dialogue. International Journal of Computational Science and Engineering, 17(1):109–117. ↗
  17. Bing Liu and Ian Lane. 2016. Joint online spoken language understanding and language modeling with recurrent neural networks. arXiv preprint arXiv:1609.01462. ↗
  18. Wolfgang Minker. 1998. Speech Understanding for Spoken Language Systems: Portability Across Domains and Languages. Hänsel-Hohenhausen. ↗
  19. Mikio Nakano, Shun Sato, Kazunori Komatani, Kyoko Matsuyama, Kotaro Funakoshi, and Hiroshi G Okuno. 2011. A two-stage domain selection framework for extensible multi-domain spoken dialogue systems. In Proceedings of the SIGDIAL 2011 Conference, pages 18–29. Association for Computational Linguistics. find
  20. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. ↗
  21. Islam Today. 2000. Alistisharat. http://www.islamtoday.net/istesharat/ index.htm. [Online; accessed 21/11/2018]. Zhuoran Wang, Hongliang Chen, Guanchun Wang, Hao Tian, Hua Wu, and Haifeng Wang. Policy learning for domain selection in an extensible multi-domain spoken dialogue system. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lichouri-2019-arabic-multi-domain-spoken-language/