Elamur Research⌕
☰
home / papers / 2015 / Towards Understanding Egyptian Arabic Dialogues

Towards Understanding Egyptian Arabic Dialogues

Paper · 2015 · Abdelrahim A Elmadany, Sherif M Abdou and Mervat Gheith

Summary

The paper addresses dialogue act (DA) classification for Egyptian Arabic. The task is to label each user utterance with the speaker's intent, which is the core of the understanding layer in a dialogue system. Before this work, Arabic DA research had covered only MSA and Tunisian dialect.

The authors built YOSR, a supervised classifier using a single multi-class SVM. They used the YamCha toolkit and framed utterance labelling as a text-chunking problem. YOSR uses no hand-built lexicons, cue lists or rules. Its features are morphological features from MADAMIRA, which needs no diacritization, plus utterance-level features.

They evaluated YOSR on JANA, a manually transcribed, segmented and annotated corpus. JANA contains spontaneous call-center dialogues and instant messages from three domains. YOSR reached an overall F1 of 70.36% across domains. The authors present this as the first reported result for understanding Egyptian dialect dialogues.

The work matters because it supplies a dialect-specific resource and baseline for an under-resourced variety. The data is noisy, with code-switching, spelling variation and heavy lexical ambiguity.

Key points

  • JANA corpus statistics:
    • roughly 3001 turns averaging 6.7 words each;
    • 4725 utterances averaging 4.3 words;
    • 20,311 words in total;
    • Egyptian dialect, from call centers, covering both spoken dialogues and IM.
  • The DA schema splits labels into three groups:
    • Request acts, e.g. Service-Question, YesNo-Question, Confirm-Question, Choice-Question.
    • Response acts, e.g. Service-Answer, Agree, Disagree.
    • Other acts, e.g. Greeting, Apology, Correct, MissUnderstandingSign, Pausing, Suggest, Promise, Warning, Offer.
  • The classifier is one multi-class SVM via YamCha, replacing a bank of one-vs-rest binary classifiers. The authors argue this cuts training and running cost.
  • The listed challenges are:
    • spontaneous-speech effects (repairs, repetitions, ellipsis, hesitations);
    • code-switching with English loanwords such as "transaction" and "active";
    • dialect deviation from MSA, e.g. EAyz/EAwz versus MSA Aryd for "want";
    • ambiguity in undiacritized text;
    • scarce resources.
  • Overall F1 is 70.36%. Planned improvements include call-center cues, first-verb-type and lemma features, context features, and handling of dialect words.

Interesting ideas

  • I1. The paper separates a turn (one speaker's whole contribution) from an utterance (a span carrying exactly one act), so a single turn can hold several acts that are labelled separately.
  • I2. Intent labelling is recast as sequence chunking and solved with one multi-class model rather than one binary detector per act, which keeps the label set coherent and training cheap.
  • I3. Features come from a morphological analyzer's output instead of hand-written cue-word lists, which makes the classifier less brittle to spelling variants and dialect forms.
  • I4. The corpus examples show self-repair inside a turn ("arrival on 12, sorry, 11 December"), where an apology marker signals that a later value replaces an earlier one.
  • I5. The authors point out that earlier DA studies often did not publish their train/test splits, which makes results hard to compare.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Speech-act tagging in prag could first split a sentence or reply into act-sized spans, so that "No, no, I have no account, but my husband works here" yields a denial plus an inform rather than one blended label.
  • I2 (frames): Speech-act and implicature labels could be predicted jointly as chunk tags over the UD-segmented text, with each chunk's label traceable to its features for explainability.
  • I3 (tokens): Lemma, POS and morphological features from crate en could serve as the inputs to downstream act classifiers, so that variant spellings or informal forms map to the same lemma and trigger the same act.
  • I4 (world): When building the world of a text, a Correct or Apology act should retract the earlier entity or state value and assert the new one, so the order of assertions is tracked rather than both values kept.
  • I5 (learn): Every speech-act evaluation set should ship with a fixed, published split and per-domain breakdowns, so that improvements from silver data or teacher tiers are measured honestly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cairo UniversityGiza · Egypt42 works · 1,350 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksDialogue systems · 624 works

Cite

Cite as Abdelrahim A Elmadany, Sherif M Abdou and Mervat Gheith. 2015. Towards Understanding Egyptian Arabic Dialogues. arXiv:1509.03208 · International Journal of Computer Applications 120(220, PP 7-12, June 2015.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5120/21390-4427 · arXiv 1509.03208

@misc{elmadany2015understanding,
  title = {{Towards Understanding Egyptian Arabic Dialogues}},
  author = {Abdelrahim A Elmadany, Sherif M Abdou and Mervat Gheith},
  year = {2015},
  doi = {10.5120/21390-4427},
  eprint = {1509.03208},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5120/21390-4427}
}

References

37 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Elmadany, A.A., Abdou, S.M., Gheith, M.: Recent Approaches to Arabic Dialogue Acts Classifications. 4th International Conferences on Natural Language Processing (NLP-2015) - Computer Science & Information Technology (CS & IT) Series 5, 117–129 (2015) find
  2. Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C.V., Meteer, M.: Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics 26, 339-373 (2000) International Journal of Computer Applications (0975 – 8887) Volume 120 – No.22, June 2015
  3. Webb, N.: Cue-Based Dialogue Act Classification. Department of Computer Science. University of Sheffield, England (2010) find
  4. Traum, D., Heeman, P.A.: Utterance units in spoken dialogue. Dialogue processing in spoken language systems, pp. 125-140. Springer, Berlin Heidelberg (1997) find
  5. Graja, M., Jaoua, M., Belguith, L.H.: Discriminative Framework for Spoken Tunisian Dialect Understanding. 2nd International Conference on Statistical Language and Speech Processing, SLSP 2014, pp. 102–110. (2013) find
  6. Pasha, A., Al-Badrashiny, M., Diab, M., Kholy, A.E., Eskander, R., Habash, N., Pooleery, M., Rambow, O., Roth, R.M.: MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. Language Resources and Evaluation Conference (LREC 2014), pp., (2014)
  7. Elmahdy, M., Rainer, G., Wolfgang, M., Slim, A.: Survey on common Arabic language forms from a speech recognition point of view. In proceeding of International conference on Acoustics (NAG-DAGA), pp. 63-66. (2009) find
  8. Zaidan, O.F., Callison-Burch, C.: Arabic dialect identification. Computational Linguistics 52, (2012)
  9. Zaghouani, W.: Critical Survey of the Freely Available Arabic Corpora. In proceeding of Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools (LREC2014). (2014) find
  10. Elmadany, A.A., Abdou, S.M., Gheith, M.: Turn Segmentation into Utterances for Arabic Spontaneous Dialogues and Instant Messages. International Journal on Natural Language Computing (IJNLC) 4, 111-123 (2015) find
  11. Elmadany, A.A., Abdou, S.M., Gheith, M.: JANA: An Arabic Human-Human Dialogues Corpus. IEEE 2 nd International Conference on Recent Trends in Information Systems (ReTIS), pp. IEEE, (2015) find
  12. Elmadany, A.A., Abdou, S.M., Gheith, M.: Arabic Inquiry-Answer Dialogue Acts Annotation Schema. IOSR Journal of Engineering (IOSRJEN) 04, 32-36 (2014)
  13. Webb, N., Hardy, H.: Data-Driven Understanding for Spoken Language American Association for Artificial (2005) Language Dialogue. find
  14. Seneff, S., Hirschman, L., Zue, V.W.: Interactive Problem Solving and Dialogue in the ATIS Domain. In proceeding of HLT '91: Proceedings of the Workshop on Speech and Natural Language, pp. 354-359. (1991) find
  15. Pellom, B., Ward, W., Hansen, J., Cole, R., Hacioglu, K., Zhang, J., Yu, X., Pradhan, S.: University of Colorado Dialog Systems for Travel and Navigation. In proceeding of HLT '01: Proceedings of the First International Conference on Human Language Technology Research. (2001) find
  16. Wahlster, W.: Verbmobil: Foundations of Speech-To-Speech Translation. Springer (2000) find
  17. Hardy, H., Strzalkowski, T., Wu, M., Ursu, C., Webb, N., Biermann, A., Inouye, R., McKenzie, A.: Data-driven strategies for an automated dialogue system. In proceeding of the 42nd Annual Meeting on Association for Computational Linguistics. (2004) find
  18. Webb, N., Hepple, M., Wilks, Y.: Dialogue Act Classification Based on Intra-Utterance Features. In proceeding of the AAAI Work-shop on Spoken Language Understanding. (2005) find
  19. Boyer, K., Ha, E., Pillips, R., Wallis, M., Vouk, M., Laster, J.: Dialogue Act Modleing in a Complex Task-Orinted Domain. In proceeding of SIGDIAL, pp. 297-305. (2010) find
  20. Samuel, K., Carberry, S., Vijay-Shanker, K.: Dialogue act tagging with transformation-based learning. In proceeding of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics. (1998)
  21. Grau, S., Sanchis, E., Castro, M., Vilar, D.: Dialogue Act Classification using a Bayesian Approach. In proceeding of 9th Conference Speech and Computer. (2004) find
  22. Shala, L., Rus, V., Graesser, A.: Automatic Speech Act Classification In Arabic. In proceeding of Subjetividad y Procesos Cognitivos Conference pp. 284-292. (2010) find
  23. Bahou, Y., Belguith, L.H., Hamadou, A.B.: Towards a Human-Machine Spoken Dialogue in Arabic. In proceeding of Workshop on HLT & NLP within the Arabic world: Arabic Language and local languages processing: Status Updates and Prospects, at the 6th Language Resources and Evaluation Conference (LREC’08). (2008) find
  24. Lhioui, C., Zouaghi, A., Zrigui, M.: A Combined Method Based on Stochastic and Linguistic Paradigm for the Understanding of Arabic Spontaneous Utterances. In proceeding of CICLing 2013, Computational Linguistics and Intelligent Text Processing Lecture Notes in Computer Science, pp. 549-558. Springer-Verlag Berlin Heidelberg, (2013) find
  25. Hijjawi, M., Bandar, Z., Crockett, K.: User’s Utterance Classification Using Machine Learning for Arabic Conversational Agents. In proceeding of 5th International Conference on Computer Science and Information Technology (CSIT), pp. 223-232. IEEE, (2013) find
  26. Hijjawi, M., Bandar, Z., Crockett, K., Mclean, D.: ArabChat: an Arabic Conversational Agent. In IJCA TM : www.ijcaonline.org International Journal of Computer Applications (0975 – 8887) Volume 120 – No.22, June 2015 proceeding of 6th International Conference on Computer Science and Information Technology (CSIT), pp. 227-237. IEEE Computer Society, (2014) find
  27. Elmadany, A.A., Abdou, S.M., Gheith, M.: A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message. International Journal on Natural Language Computing (IJNLC) 4, 75-94 (2015)
  28. Kudo, T., Matsumoto, Y.: Chunking with support vector machines. In proceeding of NAACL-01. (2001)
  29. Kudo, Y., Matsumoto, Y.: Use of Support Vector Learning for Chunk Identification In proceeding of CoNLL, pp. 142-144. (2000) find
  30. Meselhi, M.A., Bakr, H.M.A., Ziedan, I., Shaalan, K.: A Novel Hybrid Approach to Arabic Named Entity Recognition. The 10th China Workshop on Machine Translation (CWMT 2014), pp. 93–103. (2014) find
  31. Pradhan, S., Hacioglu, K., Krugler, V., Ward, W., Martin, J.H., Jurafsky, D.: Support vector learning for semantic argument classification. Machine Learning 60 Machine Learning, 11-39 (2005) find
  32. Ramshaw, L.A., Marcus, M.P.: Text Chunking Using Transformation-based Learning. In proceeding of the Third ACL Workshop on Very Large Corpora (WVLC 1995). (1995)
  33. Kim, S.N., Cavedon, L., Baldwin, T.: Classifying dialogue acts in one-on-one live chats. In proceeding of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), pp. 862–871. (2010)
  34. Ivanovic, E.: Automatic utterance segmentation in instant messaging dialogue. In proceeding of The Australasian Language Technology Workshop, pp. 241-249. (2005) find
  35. Ivanovic, E.: Automatic instant messaging dialogue using statistical models and dialogue acts. Faculty of Engineering, Computer Science and Software Engineering. University of Melbourne (2008) find
  36. Sridhara, V.K.R., Bangaloreb, S., Narayanana, S.: Combining lexical, syntactic and prosodic cues for improved online dialog act tagging. Computer Speech & Language 23, 407–422 (2009) find
  37. Eugenio, B.D., Xie, Z., Serafin, R.: Dialogue act classification, higher order dialogue structure, and instance-based learning. Dialogue and Discourse 1, 1-24 (2010) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elmadany-2015-towards-understanding-egyptian-arabic-dialogues/