Elamur Research⌕
☰
home / papers / 2020 / Empathy-driven Arabic Conversational Chatbot

Empathy-driven Arabic Conversational Chatbot

Paper · 2020 · Tarek Naous, Christian Hokayem and Hazem Hajj

Summary

Arabic had no chatbot that could notice a user's emotional state and reply with empathy, and no conversational data labeled for empathy. The authors address both gaps. They build an Arabic empathetic dialogue corpus by machine-translating the English EmpatheticDialogues dataset, in which a speaker recounts a personal situation and a listener answers with empathy. A manual check of a sample judged about 96% of the translations accurate. The corpus is too small for large Transformer models, so they train a compact Seq2Seq network instead. It uses Farasa tokenization, a two-layer bidirectional LSTM encoder, attention, and a two-layer LSTM decoder. Its settings were chosen by a validation-set search over depth, cell type, embedding size and optimizer. The best configuration uses 500-dimensional embeddings and beam search. It reaches a perplexity of 38.6 and a BLEU of 0.5. Human judges rate it 3.7 for empathy and 3.92 for fluency, but only 3.16 for relevance. The work matters as the first Arabic empathetic chatbot. It also offers a reusable low-resource recipe: translate an existing resource, then fit a model sized to the data.

Key points

  • The Arabic corpus is a translation of EmpatheticDialogues (Rashkin et al., 2018), with an estimated translation accuracy of 96% on a checked sample.
  • The model has a 2-layer BiLSTM encoder, attention over the second encoder layer, and a 2-layer LSTM decoder. It uses dropout 0.3, SGD, Farasa preprocessing, and a vocabulary of 12,900 tokens.
  • The tuning grid covered 1–3 layers, LSTM vs GRU, several embedding sizes, and SGD/Adam/Adagrad. The final model uses LSTM units with d=500.
  • Results: PPL 38.6 and BLEU 0.5. Human ratings are Empathy 3.7, Fluency 3.92 and Relevance 3.16 with beam search. Random sampling scored much lower (Empathy 2.40, Relevance 1.92, Fluency 2.80 at d=500).
  • Observed failures:
    • Beam search keeps reusing the same few stock phrases.
    • Random sampling gives longer, more varied but less coherent replies.
    • The model attaches emotion even to neutral factual questions.

Interesting ideas

  • I1. Bootstrap a missing resource by translating an existing English dataset, then audit a random sample by hand to estimate its quality before training. This gives a cheap, quantified trust level for silver data.
  • I2. Rate outputs on separate human axes (empathy, relevance, fluency) rather than one overall score. Here this showed a model that sounds fluent while drifting off topic.
  • I3. Compare decoding strategies side by side. Beam search was safe but repetitive, while sampling was varied but noisy, which makes the diversity-versus-quality trade-off visible.
  • I4. Training only on emotional dialogues made the bot respond emotionally even to plain questions. Probing a system with inputs that lack the target phenomenon exposes this kind of bias.
  • I5. Empathy is framed as first inferring the speaker's emotional state and then choosing a fitting reaction, such as comforting sadness or congratulating pride.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver training sets could carry a measured hand-audit accuracy on a sample, so each data source has a known reliability before it enters training.
  • I2 (answer): Answer evaluation could score correctness, grounding in the text, and wellformedness separately, so a fluent but ungrounded answer is caught.
  • I3 (back): Generation from the meaning graph could be checked for repeated stock phrasings across different inputs, a sign that the realizer collapses to templates.
  • I4 (learn): Neutral sentences with no irony or no emotion could serve as negative controls, confirming that classifiers do not fire everywhere they were trained to fire.
  • I5 (frames): The pragmatics layer could tag an utterance's expressed feeling as part of the speech act, and feed it to the response choice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksDialogue systems · 624 works

Cite

Cite as Tarek Naous, Christian Hokayem and Hazem Hajj. 2020. Empathy-driven Arabic Conversational Chatbot. Proceedings of the Fifth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{naous2020empathy,
  title = {{Empathy-driven Arabic Conversational Chatbot}},
  author = {Tarek Naous, Christian Hokayem and Hazem Hajj},
  booktitle = {Proceedings of the Fifth Arabic Natural Language Processing Workshop},
  year = {2020},
  doi = {10.48448/sjrf-2w66},
  url = {https://doi.org/10.48448/sjrf-2w66}
}

References

22 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16.
  2. Sarah AlHumoud, Asma Al Wazrah, and Wafa Aldamegh. 2018. Arabic chatbots: A survey. International Journal of Advanced Computer Science and Applications, 9(8):535–541. ↗
  3. Dana Abu Ali and Nizar Habash. 2016. Botta: An arabic dialect chatbot. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 208–212.
  4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France, May. European Language Resource Association. ↗
  5. Nabiha Asghar, Ivan Kobyzev, Jesse Hoey, Pascal Poupart, and Muhammad Bilal Sheikh. 2020. Generating emotionally aligned responses in dialogues using affect control theory. arXiv preprint arXiv:2003.03645. ↗
  6. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ↗
  7. Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. 2019. hULMonA: The universal language model in Arabic. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 68–77, Florence, Italy, August. Association for Computational Linguistics.
  8. Ahmed Fadhil and Ahmed AbuRa’ed. 2019. OlloBot - towards a text-based Arabic health conversational agent: Evaluation and results. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 295–303, Varna, Bulgaria, September. INCOMA Ltd. ↗
  9. Asma Ghandeharioun, Daniel McDuff, Mary Czerwinski, and Kael Rowan. 2019. Emma: An emotion-aware wellbeing chatbot. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1–7. IEEE. find
  10. Mohammad Hijjawi, Zuhair Bandar, Keeley Crockett, and David Mclean. 2014. ArabChat: an arabic conversational agent. In 2014 6th International Conference on Computer Science and Information Technology (CSIT), pages 227–237. IEEE. find
  11. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810. ↗
  12. Zhaojiang Lin, Peng Xu, Genta Indra Winata, Farhad Bin Siddique, Zihan Liu, Jamin Shin, and Pascale Fung. 2020. CAiRE: an end-to-end empathetic chatbot. In AAAI, pages 13622–13623. ↗
  13. Hussein Mozannar, Karl El Hajal, Elie Maamary, and Hazem Hajj. 2019. Neural arabic question answering. arXiv preprint arXiv:1906.05394. ↗
  14. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207. ↗
  15. Jamin Shin, Peng Xu, Andrea Madotto, and Pascale Fung. 2019. Happybot: Generating empathetic dialogue responses by improving user experience look-ahead. arXiv preprint arXiv:1906.08487. ↗
  16. Jamin Shin, Peng Xu, Andrea Madotto, and Pascale Fung. 2020. Generating empathetic responses by looking ahead the user’s sentiment. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7989–7993. IEEE. ↗
  17. Özge Nilay Yalçın and Steve DiPaola. 2018. A computational model of empathy for interactive agents. Biologically Inspired Cognitive Architectures, 26:20–25. ↗
  18. Özge Nilay Yalçın. 2020. Empathy framework for embodied conversational agents. Cognitive Systems Research, 59:123–132. ↗
  19. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243. ↗
  20. Peixiang Zhong, Yan Zhu, Yong Liu, Chen Zhang, Hao Wang, Zaiqing Nie, and Chunyan Miao. 2020. Endowing empathetic conversational models with personas. arXiv preprint arXiv:2004.12316. ↗
  21. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The design and implementation of XiaoIce, an empathetic social chatbot. Computational Linguistics, 46(1):53–93. ↗
  22. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference on Computer Vision, pages 19–27. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naous-2020-empathy-driven-arabic-conversational-chatbot/