Elamur Research⌕
☰
home / papers / 2026 / Adaptive Emotion Management in Human-robot Dialogue using Online Group Relative Policy Optimization

Adaptive Emotion Management in Human-robot Dialogue using Online Group Relative Policy Optimization

Summary

Robots that express emotion usually rely on emotion classifiers that are trained once and then frozen. Such robots cannot adjust to how a particular person reads their affective signals. Manaseryan and Kennington address this by treating emotion selection in dialogue ("emotion management") as a policy that keeps learning during live interaction from simple yes/no human judgments. Their pipeline runs on the Retico incremental framework and links several components:

  • speech recognition
  • ChildLM, a small toddler-style language model, for the robot's speech
  • a DeBERTa-v3-base emotion classifier with LoRA adapters
  • the GRED generator, which turns the chosen emotion label into a new multimodal behavior on a Cozmo robot

The classifier is first fine-tuned on synthetic dialogues. It is then updated online with Group Relative Policy Optimization (GRPO). Each turn's binary feedback is spread across a sampled group of candidate emotions, and a KL penalty keeps the policy from drifting too far from its starting point. With one tutor, approval improved steadily across sessions. Ten new users then judged the adapted robot more coherent than a non-adapted baseline. The work shows that GRPO can stay stable under sparse, noisy, real-time human feedback in embodied dialogue.

Key points

  • Policy and update rule: The emotion policy is a DeBERTa-v3-base classifier fine-tuned with LoRA. Online GRPO uses a similarity-aware reward, a small feedback buffer, asynchronous updates, and a KL penalty toward a frozen reference model.
  • Behavior generation: GRED, built on GPT-2 Medium and fine-tuned on 600 hand-designed sequences, realizes six grouped emotion categories as Cozmo behaviors (face display, sounds, driving, head and lift-arm motion). It receives only the emotion label, not the dialogue text.
  • Speech generation: ChildLM is a ~155M-parameter model trained on CHILDES. It stays fixed in all conditions, so only the emotion policy differs between systems.
  • Tutor experiment: Across 542 live interactions with one tutor, session approval rose from 66.7% to 94.9%. The same classifier without online adaptation reached 51.4%. Roughly 150 interactions were needed before rolling approval stayed above 90%.
  • User study: In a within-subjects study with N = 10, GRPO outperformed the baseline on consistent personality (4.50 vs. 3.70), in-sync emotions (3.00 vs. 2.40) and clear emotions (2.70 vs. 2.20), with equal comfort ratings. The authors flag several limitations: the small sample, a single tutor, keyboard-based feedback, and a baseline trained on synthetic data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.41

@inproceedings{manaseryan2026adaptive,
  title = {{Adaptive Emotion Management in Human-robot Dialogue using Online Group Relative Policy Optimization}},
  author = {Manaseryan, Anna and Kennington, Casey},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.41},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.41}
}

References

42 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hojjat Abdollahi, Mohammad H. Mahoor, Rohola Zandie, Jarid Siewierski, and Sara H. Qualls. 2023. Artificial emotional intelligence in socially assistive robots for older adults: A pilot study. IEEE Transactions on Affective Computing, 14(3):2020–2032. find
  2. Faruk Akgul. 2013. ZeroMQ. Packt Publishing. find
  3. Dilip Arumugam, Jun Ki Lee, Sophie Saskin, and Michael L. Littman. 2019. Deep reinforcement learning from policy-dependent human feedback. Preprint, arXiv:1902.04257. — [arXiv:1902.04257](https://arxiv.org/abs/1902.04257) find
  4. Rista Baral, Bethany Grenz, and Casey Kennington. 2025. Recognizing and generating novel emotional behaviors on two robotic platforms. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 21503–21510. find
  5. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. Preprint, arXiv:2006.03654. find our card
  6. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. find
  7. Taewoon Kim and Piek Vossen. 2021. Emoberta: Speaker-aware emotion recognition in conversation with roberta. Preprint, arXiv:2108.12009. — [arXiv:2108.12009](https://arxiv.org/abs/2108.12009) find
  8. W Bradley Knox and Peter Stone. 2009. Interactively shaping agents via human reinforcement: The tamer framework. In Proceedings of the fifth international conference on Knowledge capture, pages 9–16. find
  9. Lisa Feldman Barrett. 2017. The theory of constructed emotion: an active inference account of interoception and categorization. Soc. Cogn. Affect. Neurosci., 12(1):1–23. find
  10. Kimin Lee, Laura Smith, and Pieter Abbeel. 2021. Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre-training. Preprint, arXiv:2106.05091. — [arXiv:2106.05091](https://arxiv.org/abs/2106.05091) find
  11. Yanrong Chen and Xihan Bian. 2026. Real-time synchronized interaction framework for emotion-aware humanoid robots. Preprint, arXiv:2601.17287. — [arXiv:2601.17287](https://arxiv.org/abs/2601.17287) find
  12. Enoch Levandovsky, Anna Manaseryan, and Casey Kennington. 2025. Learning to speak like a child: Reinforcing and evaluating a child-level generative language model. In Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 370–382. find
  13. Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2023. Deep reinforcement learning from human preferences. Preprint, arXiv:1706.03741. find
  14. Christian Arzate Cruz and Takeo Igarashi. 2021. A survey on interactive reinforcement learning: Design principles and open challenges. Preprint, arXiv:2105.12949. — [arXiv:2105.12949](https://arxiv.org/abs/2105.12949) find
  15. Kerstin Fischer, Malte Jung, Lars Christian Jensen, and Maria Vanessa aus der Wieschen. 2019. Emotion expression in hri – when and why. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 29–38. find
  16. Marialejandra García-Corretjer, Raquel Ros, and Roger Mallol. 2022. Empathy as an engaging strategy in social robotics: a pilot study. User Modeling and User-Adapted Interaction, 33:1–39. find
  17. Milica Gašić, Catherine Breslin, Matthew Henderson, Dongho Kim, Martin Szummer, Blaise Thomson, Pirros Tsiakoulis, and Steve Young. 2013. On-line policy optimisation of bayesian spoken dialogue systems via human interaction. In 2013 IEEE International Conference on Acoustics, Speech and Signal Proc… find
  18. James MacGlashan, Mark K Ho, Robert Loftin, Bei Peng, Guan Wang, David L Roberts, Matthew E Taylor, and Michael L Littman. 2017. Interactive learning from policy-dependent human feedback. In International conference on machine learning, pages 2285–2294. PMLR. find
  19. Brian Macwhinney. 2000. The childes project: tools for analyzing talk. Child Language Teaching and Therapy, 8. find
  20. Karthik Mahadevan, Jonathan Chien, Noah Brown, Zhuo Xu, Carolina Parada, Fei Xia, Andy Zeng, Leila Takayama, and Dorsa Sadigh. 2024. Generative expressive robot behaviors using large language models. In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pages 482–… find
  21. Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6818–6825. find
  22. Anna Manaseryan, Porter Rigby, Brooke Matthews, Catherine Henry, Josue Torres-Fonseca, Ryan Whetten, Enoch Levandovsky, and Casey Kennington. 2025. rrSDS 2.0: Incremental, modular, distributed, multimodal spoken dialogue with robotic platforms. In Proceedings of the 26th Annual Meeting of the Specia… find
  23. David McNeill and Casey Kennington. 2019. Predicting human interpretations of affect and valence in a social robot. In Proceedings of Robotics: Science and Systems, FreiburgimBreisgau, Germany. find
  24. Thilo Michael. 2020. Retico: An incremental framework for spoken dialogue systems. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 49–52, 1st virtual meeting. Association for Computational Linguistics. find
  25. Chinmaya Mishra, Rinus Verdonschot, Peter Hagoort, and Gabriel Skantze. 2023. Real-time emotion generation in human-robot dialogue using large language models. Frontiers in Robotics and AI, Volume 10 -2023. find
  26. Lara Toledo Cordeiro Ottoni and Jés de Jesus Fiais Cerqueira. 2024. A systematic review of human– robot interaction: the use of emotions and the evaluation of their performance. International Journal of Social Robotics, 16(11):2169–2188. find
  27. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train… find
  28. Punya Syon Pandey and Zhijing Jin. 2026. Binaryppo: Efficient policy optimization for binary classification. Preprint, arXiv:2602.02708. — [arXiv:2602.02708](https://arxiv.org/abs/2602.02708) find
  29. Patrícia Pereira, Helena Moniz, and Joao Paulo Carvalho. 2024. Deep emotion recognition in textual conversations: A survey. Preprint, arXiv:2211.09172. — [arXiv:2211.09172](https://arxiv.org/abs/2211.09172) find
  30. Sarah Plane, Ariel Marvasti, Tyler Egan, and Casey Kennington. 2018. Predicting perceived age: Both language ability and appearance are important. In Proceedings of SigDial. find
  31. Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and recent advances. Preprint, arXiv:1905.02947. — [arXiv:1905.02947](https://arxiv.org/abs/1905.02947) find
  32. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. arXiv preprint. find our card
  33. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. find
  34. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300. find our card
  35. Ruth Stock-Homburg. 2021. Survey of emotions in human–robot interactions: Perspectives from robotic psychology on 20 years of research. International Journal of Social Robotics, 14. find
  36. Pei-Hao Su, Milica Gasic, Nikola Mrkšić, Lina M Rojas Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016a. On-line active reward learning for policy optimisation in spoken dialogue systems. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin… find
  37. Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016b. Continuously learning neural dialogue management. Preprint, arXiv:1606.02689. find
  38. Christian Tamantini, Alessandro Umbrico, Francesca Fracasso, Gloria Beraldo, Gabriella Cortellessa, and Andrea Orlandini. 2026. Adapting virtual agent interaction style with reinforcement learning to enhance affective engagement. Frontiers in Digital Health, Volume 7 - 2025. find
  39. NVIDIA The NeMo Data Designer Team. 2025. Nemo data designer: A framework for generating synthetic data from scratch or based on your own seed data. https://github.com/NVIDIA-NeMo/ DataDesigner. GitHub Repository. ↗
  40. Josue Torres-Fonsesca and Casey Kennington. 2022. HADREB: Human appraisals and (English) descriptions of robot emotional behaviors. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5739–5748, Marseille, France. European Language Resources Association. find
  41. Peisong Wang, Ruotian Ma, Bang Zhang, Xingyu Chen, Zhiwei He, Kang Luo, Qingsong Lv, Qingxuan Jiang, Zheng Xie, Shanyi Wang, Yuan Li, Fanghua Ye, Jian Li, Yifan Yang, Zhaopeng Tu, and Xiaolong Li. 2025a. Rlver: Reinforcement learning with verifiable emotion rewards for empathetic agents. Preprint, a… — [arXiv:2507.03112](https://arxiv.org/abs/2507.03112) find
  42. Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Ellie Evans, Daniel Egert, Hoo-Chang Shin, Felipe Soares, Yi Dong, and Oleksii Kuchaiev. 2025b. Rlbff: Binary flexible feedback to bridge between human feedback & verifiable rewards. Preprint, arXiv:2509.21319. — [arXiv:2509.21319](https://arxiv.org/abs/2509.21319) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/manaseryan-2026-adaptive-emotion-management-human-robot/