Elamur Research⌕
☰
home / papers / 2024 / ABLE: Personalized Disability Support with Politeness and Empathy Integration

ABLE: Personalized Disability Support with Politeness and Empathy Integration

Paper · 2024 · Kshitij Mishra, Manisha Burja and Asif Ekbal

Summary

The paper addresses the lack of personalization and warmth in conversational support for people with physical disabilities. The authors built PerPDSCD, a large synthetic dialogue corpus in which a doctor-like agent talks with users. They produced it with GPT-3.5, refined the prompts with domain experts, and added human intervention. Every dialogue is annotated with the user's gender, age group and an OCEAN personality profile, and the agent's turns carry politeness and empathy information.

On this corpus they trained ABLE, a support system built on the Phi-2 language model. It is optimized with reinforcement learning, using a reward that combines four reward models. Together these push responses to fit the user's persona while keeping suitable politeness and empathy. Automatic and human evaluation covered persona consistency, politeness and empathy accuracy, perplexity and coherence, and the authors report that ABLE delivers effective tailored support. They present it as the first persona-oriented assistant for physical disability support, a domain that so far relied on generic or rule-based systems.

Key points

  • PerPDSCD contains 18,026 dialogues annotated with user profiles, generated with GPT-3.5 plus human intervention.
  • The corpus covers 13 support topics and 14 disability or issue types:
    • Topics include mobility aids, pain management, employment and education, and parenting with disabilities.
    • Disability types include spinal cord injuries, cerebral palsy, chronic pain and aging-related conditions.
  • User profiles vary along three dimensions:
    • Gender: male or female.
    • Age: younger, middle-aged or older.
    • Personality: one of 19 expert-validated OCEAN trait combinations, chosen to reflect configurations common in real populations.
  • ABLE fine-tunes Phi-2 with reinforcement learning, using a reward built from four reward models covering persona, politeness and empathy alignment.
  • The reported limitations are:
    • possible hallucination, which calls for knowledge grounding;
    • off-context replies after very short user turns such as "yes", "no" or "is it?";
    • uneven politeness and empathy;
    • loss of coherence in long dialogues.

Interesting ideas

  • I1. Instead of crossing all five OCEAN traits blindly, the authors keep only 19 plausible trait combinations validated by experts, so synthetic data stays realistic without exploding in size.
  • I2. The reward is decomposed into several separate scorers, one per desired property (persona fit, politeness, empathy), rather than one monolithic judge, so each property can be steered and diagnosed separately.
  • I3. The LLM prompt template for data generation was refined iteratively with domain-expert feedback before mass generation, which is cheap quality control at the source.
  • I4. Very short user turns ("yes", "is it?") were identified as a distinct failure class, since their meaning depends entirely on the previous turn.
  • I5. Evaluation measures whether the requested attribute was actually realized (politeness or empathy accuracy), alongside fluency and coherence, rather than fluency alone.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when generating silver or teacher data across attribute grids (genre, register, speaker type), restrict to plausible combinations to avoid training on unnatural mixtures.
  • I2 (back): text generation from the meaning graph could be scored by independent checkers (faithfulness to graph, register, fluency), with each failure traced to a specific checker.
  • I3 (learn): LLM teacher tiers could use a short expert-review loop on the prompt itself before producing large silver batches.
  • I4 (frames): treat bare confirmations and elliptical questions as speech acts resolved against the prior turn's frame, and add them as an explicit test category.
  • I5 (answer): answer quality checks could separately verify that each requested property of an answer is present, giving explainable pass/fail per property.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology PatnaPatna · India21 works · 374 citations🎓Indian Institute of Technology JodhpurJodhpur · India3 works · 3 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHealth Professions · 69 worksOccupational Therapy · 9 works

Cite

Cite as Kshitij Mishra, Manisha Burja and Asif Ekbal. 2024. ABLE: Personalized Disability Support with Politeness and Empathy Integration. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.emnlp-main.1252

@inproceedings{mishra2024able,
  title = {{ABLE: Personalized Disability Support with Politeness and Empathy Integration}},
  author = {Kshitij Mishra, Manisha Burja and Asif Ekbal},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.1252},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.1252}
}

References

58 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Adams and D. Brown. 2019. Personality-aware conversational agents: A survey. IEEE Transactions on Human-Machine Systems. find
  2. James R Alleman. 2002. Online counseling: The internet and mental health treatment. Psychotherapy: Theory, Research, Practice, Training, 39(2):199. ↗
  3. G. Brown and H. Lee. 2018. Enhancing user engagement through personalized conversations. Journal of Human-Computer Interaction. find
  4. Penelope Brown and Stephen C. Levinson. 1987. Politeness: Some universals in language usage. Cambridge University Press. ↗
  5. Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, Jennifer C Lai, and Robert L Mercer. 1992. An estimate of an upper bound for the entropy of english. Computational Linguistics, 18(1):31–40. ↗
  6. Lin Cai, Jiaxin He, Yibo Wu, and Xuji Jia. 2023. The relationship between big five personality and quality of life of people with disabilities: The mediating effect of social support. Frontiers in psychology, 13:1061455. ↗
  7. Inha Cha, Sung-In Kim, Hwajung Hong, Heejeong Yoo, and Youn-kyung Lim. 2021. Exploring the use of a voice-based conversational agent to empower adolescents with autism spectrum disorder. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1–15. find
  8. X. Chen and Y. Wang. 2020. Conversational ai for healthcare: A review. IEEE Journal of Biomedical and Health Informatics. find
  9. Paul T. Jr. Costa and Robert R. McCrae. 1991. The neo personality inventory: Using the five-factor model in counseling. Journal of Counseling & Development, 69(4):367–372. find
  10. Laura Dennison, Leanne Morrison, Gemma Conway, Lucy Yardley, et al. 2013. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. Journal of medical Internet research, 15(4):e2583. ↗
  11. Sandeul Ha, Seung Hee Ho, Young-Hyeon Bae, Miny-oung Lee, Ju Hee Kim, Ju Han Kim, and Jisan Lee. 2023. Digital health equity and tailored health care service for people with disability: User-centered design and usability study. Journal of Medical Internet Research, 25:e50029. ↗
  12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. ↗
  13. L. Huang and Q. Liu. 2021. Empathy modeling in conversational systems using reinforcement learning. Journal of Artificial Intelligence Research. find
  14. Syed Mahmudul Huq, Rytis Maskeliūnas, and Robertas Damaševičius. 2022. Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: A systematic review. Disability and Rehabilitation: Assistive Technology, pages 1–20. ↗
  15. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. ↗
  16. Kathryn Johnson and Justin Jacob. 2017. Support for individuals with disabilities: Overview and recommendations. Journal of Disability Policy Studies, 27(4):238–245. find
  17. M. Johnson and R. Adams. 2017. The role of politeness and empathy in conversational systems. IEEE Transactions on Affective Computing. find
  18. M. Johnson and R. Smith. 1998. Assistive technology for individuals with physical disabilities: A review. Journal of Rehabilitation Engineering, 5(3):123–135. find
  19. Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Reza-zadegan, Agustina Briatore, and Enrico Coiera. 2019. The personalization of conversational agents in health care: systematic review. Journal of medical Internet research, 21(11):e15360. ↗
  20. S. Lee and H. Tan. 2020. Fostering supportive environments through empathetic conversations. Journal of Artificial Intelligence Research. find
  21. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463. ↗
  22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  23. Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In International Conference on Learning Representations. ↗
  24. C. F. Martinez-Cruz, J. M. Diez-Anton, and J. J. Gomez-Sanz. 2020. User model for providing personalized recommendations in web accessibility evaluation. Applied Sciences, 10(11):3867. find
  25. Behzad Karami Matin, Heather J Williamson, Ali Kazemi Karyani, Satar Rezaei, Moslem Soofi, and Shahin Soltani. 2021. Barriers in access to healthcare for women with disabilities: a systematic review in qualitative studies. BMC women’s health, 21:1– 23. ↗
  26. R. R. McCrae and P. T. Jr. Costa. 1992. Personality in adulthood: A five-factor theory perspective. Guilford Press. find
  27. Robert R. McCrae and Paul T. Jr. Costa. 2008. Empirical and theoretical status of the five-factor model of personality traits. The Sage Handbook of Personality Theory and Assessment, 1:273–294. ↗
  28. Robert R. McCrae, Antonio Terracciano, Anu Realo, and Juri Allik. 2007. On the validity of culture-level personality and stereotypes scores. Journal of Personality, 75(5):917–947. find
  29. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. ↗
  30. J. Miller and T. Lee. 2020. Advances in conversational ai: A review. AI Communications. find
  31. K. Miller and M. Wilson. 2021. Understanding variability in user personalities during conversations. Journal of Conversational AI. find
  32. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022a. Please be polite: Towards building a politeness adaptive dialogue system for goal-oriented conversations. Neurocomputing, 494:242–254. ↗
  33. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2024. Please donate to save a life: Inducing politeness to handle resistance in persuasive dialogue agents. IEEE/ACM Transactions on Audio, Speech, and Language Processing. ↗
  34. Kshitij Mishra, Priyanshu Priya, Manisha Burja, and Asif Ekbal. 2023a. e-therapist: I suggest you to cultivate a mindset of positivity and nurture uplifting thoughts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13952–13967. find
  35. Sarah Masud Preum, Sirajum Munir, Meiyi Ma, Mohammad Samin Yasar, David J Stone, Ronald Williams, Homa Alemzadeh, and John A Stankovic. 2021. A review of cognitive assistants for healthcare: Trends, prospects, and future directions. ACM Computing Surveys (CSUR), 53(6):1–37. ↗
  36. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  37. Azlaan Mustafa Samad, Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022. Empathetic persuasion: reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 844–856. find
  38. Theresa Schachner, Roman Keller, and Florian v Wan-genheim. 2020. Artificial intelligence-based conversational agents for chronic conditions: systematic literature review. Journal of medical Internet research, 22(9):e20701. find
  39. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. ↗
  40. Howard B Smith and Gail P Robinson. 1995. Mental health counseling: Past, present, and future. Journal of Counseling & Development, 74(2):158–162. ↗
  41. Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023b. Pal to lend a helping hand: Towards building an emotion adaptive polite and empathetic counseling conversational agent. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12254– 12271.
  42. Ronnie Smith and Mauro Dragone. 2023. Generalisable dialogue-based approach for active learning of activities of daily living. ACM Transactions on Interactive Intelligent Systems, 13(3):1–37. ↗
  43. Kshitij Mishra, Azlaan Mustafa Samad, Palak Totala, and Asif Ekbal. 2022b. Pepds: A polite and empathetic persuasive dialogue system for charity donation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 424–440.
  44. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. ↗
  45. Joao Luis Zeni Montenegro, Cristiano André da Costa, and Rodrigo da Rosa Righi. 2019. Survey of conversational agents in health. Expert Systems with Applications, 129:56–67. ↗
  46. World Health Organization. 2021. Disability and health. World Health Organization. find
  47. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744. ↗
  48. Monika Parchomiuk. 2019. Teacher empathy and attitudes towards individuals with disabilities. International Journal of Disability, Development and Education, 66(1):56–69. ↗
  49. W. Tan and Y. Zhao. 2020. Reinforcement learning for adaptive politeness in conversational systems. Neural Computing and Applications. find
  50. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar San-seviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct distillation of lm alignment. ↗
  51. H. Wang and G. Zhang. 2022. Neural approaches for conversational ai: Current trends and future prospects. Journal of Artificial Intelligence Research. find
  52. J. Wang and S. Li. 2018. Personalized support systems for individuals with physical disabilities. Journal of Assistive Technologies, 10(2):79–91. find
  53. J. Wang and L. Zhang. 2019. Personalization in conversational systems: A review. ACM Transactions on Interactive Intelligent Systems. find
  54. Q. Wang and S. Li. 2020. Incorporating personality traits in conversational systems. ACM Transactions on Interactive Intelligent Systems. find
  55. Nirmalie Wiratunga, Kay Cooper, Anjana Wijekoon, Chamath Palihawadana, Vanessa Mendham, Ehud Reiter, and Kyle Martin. 2020. Fitchat: conversational artificial intelligence interventions for encouraging physical activity in older adults. arXiv preprint arXiv:2004.14067. ↗
  56. Qingyang Wu, Yichi Zhang, Yu Li, and Zhou Yu. 2021. Alternating recurrent dialog model with large-scale pre-trained language models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1292–1301. find
  57. Q. Zhang and W. Liu. 2021. Recent advances in conversational systems: A survey. Journal of Intelligent Information Systems. find
  58. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mishra-2024-able-personalized-disability-support-politeness/