Elamur Research⌕
☰
home / papers / 2015 / A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message

A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message

Paper · 2015 · AbdelRahim A. Elmadany, Sherif M. Abdou and Mervat Gheith

Summary

The authors review what is needed to build the language-understanding part of Arabic dialogue systems, covering both spoken conversations and written chat or instant messages. Most dialogue resources and tools target English and other Indo-European languages. Arabic is harder for several reasons: diacritics are usually omitted, MSA differs from the spoken dialects, and those dialects have no codified written grammar. The paper places language understanding in a standard pipeline: input, optional ASR, LU, dialogue management, NLG, and output. It then organises the field into four building blocks: a dialogue act annotation schema, a dialogue corpus, segmentation of turns into utterances, and dialogue act classification. It also sets out its terminology. Dialogue acts derive from Searle's speech-act classes, and a turn (one speaker's talking time) is distinguished from an utterance (a unit carrying a single act). The well-known schemas it reviews include MapTask, VERBMOBIL, DAMSL and SWITCHBOARD-DAMSL. As a survey, it reports no new experimental result. Its value is a structured map of the techniques, schemas and corpora that Arabic dialogue-act work can build on.

Key points

  • The survey identifies four required components for Arabic dialogue LU: an annotation schema, a corpus, utterance segmentation and dialogue act classification.
  • Arabic varieties are grouped as Classic, MSA and dialectal. Six dialect groups are recognised: Gulf, Iraqi, Levantine, Egyptian and Maghrebi, plus a catch-all "other" category. Egyptian is described as the most widely understood dialect.
  • Schema sizes differ widely across projects:
    • MapTask uses 12 acts, split into initiating and response moves.
    • VERBMOBIL used a 43-act hierarchy in its first phase and an 18-act hierarchy in its second, travel-planning phase.
    • SWITCHBOARD-DAMSL has 220 acts, which makes manual annotation difficult.
  • An utterance unit follows Traum and Heeman's criteria: a single speaker, syntactic or semantic completeness, one speech act, one intonational phrase, and pause boundaries.
  • An example Egyptian-dialect banking turn from Elmadany et al. (2014) splits into four utterances: an apology, a request to ask, wanting to open an account, and asking about procedures.

Interesting ideas

  • I1. The paper decomposes understanding into a fixed checklist of schema, corpus, segmenter and classifier, so each resource gap is visible and can be tackled separately.
  • I2. It separates "turn" from "utterance": one speaker turn can hold several acts, and segmentation is a distinct step before act labelling.
  • I3. It notes that a richer label set is not automatically better, since a 220-act scheme proved hard to annotate consistently and VERBMOBIL shrank from 43 to 18 acts when the domain changed.
  • I4. It groups acts into initiating and response moves, as MapTask does, which ties each act to its role in the exchange and not only to its surface form.
  • I5. It uses Traum and Heeman's multi-cue definition of utterance boundaries, where completeness, single act, single speaker and pause each act as independent evidence.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act handling in prag could be audited against the same four-part checklist, so that missing labelled data or missing segmentation is named explicitly rather than hidden.
  • I2 (frames): prag could split a sentence or turn into act-bearing units first (e.g. "excuse me" + "I want to ask" + request), then assign one speech act per unit with a traceable boundary reason.
  • I3 (learn): when LLM teacher tiers or witnesses produce speech-act silver labels, a small coarse inventory with optional refinement should give higher agreement than a fine flat one.
  • I4 (answer): QA readers over dialogues in tales could pair each question or request with its reply move, helping decide whether a character's question was actually answered.
  • I5 (tokens): sentence and clause segmentation could log which cues fired at each boundary, keeping the splitting decision explainable instead of relying on punctuation alone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cairo UniversityGiza · Egypt42 works · 1,350 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksDialogue systems · 624 works

Cite

Cite as AbdelRahim A. Elmadany, Sherif M. Abdou and Mervat Gheith. 2015. A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message. arXiv:1505.03084 · International Journal on Natural Language Computing (IJNLC) Vol. 4, No.2,April 2015.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5121/ijnlc.2015.4206 · arXiv 1505.03084

@misc{elmadany2015survey,
  title = {{A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message}},
  author = {AbdelRahim A. Elmadany, Sherif M. Abdou and Mervat Gheith},
  year = {2015},
  doi = {10.5121/ijnlc.2015.4206},
  eprint = {1505.03084},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5121/ijnlc.2015.4206}
}

References

62 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al-Badrashiny, M. 2009. Autmmatic Diacritizer for Arabic text. MSc in Electronics & Eletrical Communications. Faculty of Enginnering, Cairo University. find
  2. Alexandersson, J., et al. 1998. Dialogue Acts in VERBMOBIL- 2 (second edition). VM report 226 , DFKI GmbH, Universities of Berlin, Saarbrcken and Stuttgart. find
  3. Allen, J., and M. Core. 1997. Draft of DAMSL: Dialog Act Markup in Several Layers. find
  4. Anderson, A., et al. 1991. The HCRC Map Task Corpus. Language and Speech, 34(351-366. ↗
  5. Ang, J., et al. 2005. Automatic Dialog Act Segmentation and Classification in Multiparty Meetings. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '05).1061 - 1064. find
  6. Bahou, Y., et al. 2008. Towards a Human-Machine Spoken Dialogue in Arabic. In Proceedings of Workshop on HLT & NLP within the Arabic world: Arabic Language and local languages processing: Status Updates and Prospects, at the 6th Language Resources and Evaluation Conference (LREC‟08). Marrakech, Maroc. find
  7. Belguith, L., et al. 2005. Segmentation de textes arabes basée sur l'analyse contextuelle des signes de ponctuations et de certaines particules. In Proceedings of 12th Conference on Natural Language Processing (TALN‟2005).451-456. find
  8. Boyer, K., et al. 2010. Dialogue Act Modleing in a Complex Task-Orinted Domain. In Proceedings of SIGDIAL. Tokyo, Japan:297-305. find
  9. Brill, E. 1995. Transformation-based error-driven learning and natural language processing: A case find
  10. Bunt, H. 1994. Context and Dialogue Control. , 1. THINK, 3(19-31. find
  11. Bunt, H., et al. 2010. Towards an ISO standard for dialogue act annotation. In Proceedings of the International Conference on Language Resources and Systems (LREC 2010). Malta, Italy:2548–2558.
  12. Bunt, H., et al. 2012. ISO 24617-2: A semantically-based standard for dialogue annotation. In Proceedings of Proceedings of the International Conference on Language Resources and Evaluation (LREC). Istanbul, Turkey.
  13. Bunt, H., and Y. Girard. 2005. Designing an Open, Multidimensional Dialogue Act Taxonomy. In Proceedings of the 9th Workshop on the Semantics and Pragmatics in Dialogue. find
  14. Canavan, A., et al. 1997. CALLHOME Egyptian Arabic Speech. in L. D. Consortium ed Philadelphia find
  15. Carberry, S., and L. Lambert. 1999. A Process Model for Recognizing Communicative Acts and Modeling Negotiation Sub-dialogues. Computational Linguistics. find
  16. Core, M., and J. Allen. 1997. Coding Dialogs with the DAMSL annotation scheme.AAAI Fall Symposium on Communicative Action in Humans and Machines. MIT, Cambridge, MA. find
  17. Core, M., et al. 1998. Report of The Third Workshop of the Discours e Resource Initiative Chiba University, Chiba, Japan. find
  18. Dbabis, S. B., et al. 2012. Dialogue Acts Annotation Scheme within Arabic discussion. In Proceedings of SemDial 2012 The 16th workshop on semantics and pragmatics of dialogue Sorbonne, Paris, France. find
  19. Diab, M., and N. Habash. 2007. Arabic Dialect Processing Tutorial. In Proceedings of The Human Language Technology Conference of the North American. Rochester find
  20. Diab, M., et al. 2004. Automatic Tagging of Arabic Text: From raw text to Base Phrase Chunks. In Proceedings of HLT/NAACL. Boston.
  21. Ding, L., and C. Zong. 2003. Utterance segmentation using combined approach based on bidirectional N-gram and maximum entropy. Proceedings of the second SIGHAN workshop on Chinese language processing. Association for Computational Linguistics, 17( find
  22. El-Shayeb, M., et al. 2007. ArabicSeg: An Arabic News story Segmentation System. In Proceedings of Third International Computer Conference Cairo University (ICENCO 2007). Cairo, Egypt. find
  23. Elmadany, A. A., et al. 2014. Arabic Inquiry-Answer Dialogue Acts Annotation Schema. IOSR Journal of Engineering (IOSRJEN), 04(12-V2):32-36. ↗
  24. Georgila, K., et al. 2009. Automatic Annotation of Context and Speech Acts for Dialogue Corpora. Journal of Natural Language Engineering:315-353. ↗
  25. Graja, M., et al. 2010. Lexical study of a spoken corpus in Tunisian dialect.The International Arab Conference on Information Technology (ACIT 2010). Benghazi, Libya. find
  26. Graja, M., et al. 2013. Discriminative Framework for Spoken Tunisian Dialect Understanding.2nd International Conference on Statistical Language and Speech Processing, SLSP 2014.102–110. find
  27. Grau, S., et al. 2004. Dialogue Act Classification using a Bayesian Approach. In Proceedings of 9th Conference Speech and Computer. find
  28. Hardy, H., et al. 2004. Data-driven strategies for an automated dialogue system. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. find
  29. Hijjawi, M., et al. 2013. User‟s Utterance Classification Using Machine Learning for Arabic Conversational Agents. In Proceedings of 5th International Conference on Computer Science and Information Technology (CSIT).223-232. find
  30. Hijjawi, M., et al. 2014. ArabChat: an Arabic Conversational Agent. In Proceedings of 6th International Conference on Computer Science and Information Technology (CSIT).227-237. find
  31. Ivanovic, E. 2005. Automatic utterance segmentation in instant messaging dialogue. In Proceedings of The Australasian Language Technology Workshop.241-249. find
  32. Jekat, S., et al. 1995. Dialogue Acts in VERBMOBIL. VM report 65 Universities of Berlin, Saarbrcken and Stuttgart. ↗
  33. Ji, G., and J. Bilmes. 2005. Dialog Act Tagging Using Graphical Models. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '05). find
  34. Ji, G., and J. Bilmes. 2006. Backo Model Training using Partially Observed Data: Application to Dialog Act Tagging. In Proceedings of the Human Language Technology/ American chapter of the Association for Computational Linguistics (HLT/NAACL'06). find
  35. Jurafsky, D., et al. 1997. Automatic Detection of Discourse Structure for Speech Recognition and Understanding. IEEE Workshop on Speech Recognition and Understanding. Santa Barbara. ↗
  36. Jurafsky, D., et al. 1998. Switch-board Discourse Language Modeling Project Final Report. In Proceedings of ResearchNote 30, Center for Language and Speech Processing. Johns Hopkins University, Baltimore. find
  37. Keskes, I., et al. 2012. Clause-based Discourse Segmentation of Arabic Texts. In Proceedings of The eighth international conference on Language Resources and Evaluation (LREC). Istanbul.
  38. Khalifa, I., et al. 2011. Arabic Discourse Segmentation Based on Rhetorical Methods. International Journal of Electric & Computer Sciences IJECS-IJENS, 11(01). find
  39. Lafferty, J., et al. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of International Conference on Machine Learning (ICML).282-289. find
  40. Lee, C., et al. 2010. Recent Approaches to Dialog Management for Spoken Dialog Systems. Computing Science and Engineering, 4(1):1-22. ↗
  41. Lhioui, C., et al. 2013. A Combined Method Based on Stochastic and Linguistic Paradigm for the Understanding of Arabic Spontaneous Utterances. In Proceedings of CICLing 2013, Computational Linguistics and Intelligent Text Processing Lecture Notes in Computer Science. Samos, Greece, 7817(2):549-558. find
  42. Nagata, M., and T. Morimoto. 1994. First Steps Towards Statistical Modeling of Dialogue to Predict the Speech Act Type of the Next Utterance. In Proceedings of Speech Communication. ↗
  43. Neifar, W., et al. 2014. Implementation of a Symbolic Method for the Tunisian Dialect Understanding. In Proceedings of 5th International Conference on Arabic Language Processing (CITALA‟2014). Oujda, Maroc. find
  44. Niedermair, G. 1992. Linguistic Modeling in the Context of Oral Dialogue. In Proceedings of International Conference on Spoken Language Processing (ICSLP'92). Ban, Canada: 63-638. find
  45. Pellom, B., et al. 2001. University of Colorado Dialog Systems for Travel and Navigation. In Proceedings of HLT '01: Proceedings of the First International Conference on Human Language Technology Research. USA. ↗
  46. Prasad, R., and M. Walker. 2002. Training a Dialogue Act Tagger for Humna-Human and Human-Computer Travel Dialogues. In Proceedings of the 3rd SIGdial workshop on Discourse and Dialogue. Philadelphia, Pennsylvania. find
  47. Reithinger, N., and M. Klesen. 1997. Dialogue Act Classification Using Language Models. In Proceedings of EuroSpeech. find
  48. Seneff, S., et al. 1991. Interactive Problem Solving and Dialogue in the ATIS Domain. In Proceedings of HLT '91: Proceedings of the Workshop on Speech and Natural Language. USA:354-359. find
  49. Shala, L., et al. 2010. Automatic Speech Act Classification In Arabic. In Proceedings of Subjetividad y Procesos Cognitivos Conference 14(2):284-292. find
  50. Shriberg, E., et al. 2004. The ICSI Meeting Recorder Dialog Act (MRDA) Corpus. In Proceedings of Special Interest Group on Discourse and Dialogue (SIGdial). Boston, USA. ↗
  51. Stolcke, A., et al. 2000. Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics, 26(3):339-373. ↗
  52. Stolcke, A., and E. Shriberg. 1996. Automatic linguistic segmentation of conversational speech. In Proceedings of The Fourth International Conference on Spoken Language Processing (LCSLP 96). 2(1005-1008. find
  53. Touir, A. A., et al. 2008. Semantic-Based Segmentation of Arabic Texts. . Information Technology Journal, 7(7). find
  54. Traum, D., and P. A. Heeman. 1997. Utterance units in spoken dialogue. Dialogue processing in spoken language systems Springer. Berlin Heidelberg,125-140. find
  55. Wahlster, W. 2000. Verbmobil: Foundations of Speech-To-Speech Translation Springer. find
  56. Webb, N. 2010. Cue-Based Dialogue Act Classification. Ph.D. dissertation. University of Sheffield, England. find
  57. Webb, N., and H. Hardy. 2005. Data-Driven Language Understanding for Spoken Language Dialogue American Association for Artificial. find
  58. Webb, N., et al. 2005a. Dialogue Act Classification Based on Intra-Utterance Features. In Proceedings of the AAAI Work-shop on Spoken Language Understanding. find
  59. Webb, N., et al. 2005b. Empirical Determination of Thresholds for Optimal Dialogue Act Classification. In Proceedings of the Ninth Workshop on the Semantics and Pragmatics of Dialogue. find
  60. Zaghouani, W. 2014. Critical Survey of the Freely Available Arabic Corpora. In Proceedings of Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools (LREC2014). find
  61. Zaidan, O. F., and C. Callison-Burch. 2012. Arabic dialect identification. Computational Linguistics, 52(1). ↗
  62. Zimmermann, M., et al. 2005. Toward Joint Segmentation and Classification of Dialog Acts in Multiparty Meetings. In Proceedings of Proc. Multimodal Interaction and Related Machine Learning Algorithms Workshop (MLMI–05). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elmadany-2015-survey-arabic-dialogues-understanding-spontaneous/