Elamur Research⌕
☰
home / papers / 2026 / Mixed-Initiative Dialogue Management for Human-Virtual Agents Interaction in Forum Theatre Inspired Training

Mixed-Initiative Dialogue Management for Human-Virtual Agents Interaction in Forum Theatre Inspired Training

Paper · 2026 · Samuel Otofa, Yacine Zerenini, Frederic Bechet et al.

Summary

The paper presents a virtual reality demonstrator for anti-discrimination training. It is modelled on Augusto Boal's forum theatre, in which spectators step into a scene to try alternative reactions. The use case is everyday sexism at work: a male supervisor questions a female employee's ability to lead a team. The user goes through three phases:

  • Witness: watching two autonomous virtual agents act out the scene.
  • Discussion: analysing the problematic behaviours by talking with a virtual character.
  • Confrontation: replaying the scene as the targeted person and trying to change its outcome.

The core technical contribution is a mixed-initiative dialogue controller. It lets an LLM handle open conversation but keeps it under an explicit dialogue state. That state records which discriminatory situations still need to be discussed. Scripted sequences can take over when needed, for explanations, legal context, or recovery from sensitive input. Agent behaviour comes from a motion-captured corpus of forum theatre performances. The paper reports no evaluation yet. Its value is as an architecture that balances flexibility with safety for dialogue on sensitive topics.

Key points

  • Pipeline: Google Speech API for ASR, inworld-tts-1-max for speech synthesis, Unity for VR animation, and Gemini-2.5-flash for generating turns.
  • Dialogue controller: a Contextual Spoken Language Understanding (CSLU) module does four things:
    • tracks which situations have been discussed;
    • triggers scripted sequences;
    • detects when a stage is finished;
    • flags harmful output from the user or the LLM, then starts a corrective script or ends the session.
  • LLM prompt and output: the prompt combines a persona, a stage description with state labels, behaviour guidelines, and stage-specific goals. The LLM returns JSON containing the avatar's utterance and any dialogue-state labels it detected.
  • Animation: built from the corpus of Ochs et al. (2023), with actors from NextLevel, using iClone, OpenFace, Mixamo and SALSA lip-sync. It includes 8 idle, 10 listening, 9 talking, 19 facial and 16 head animations.
  • Future work: extending the system to other forms of bias and assessing engagement and learning through surveys.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Samuel Otofa, Yacine Zerenini, Frederic Bechet, Benoit Favre, Jean-Marie Pergandi and Magalie Ochs. 2026. Mixed-Initiative Dialogue Management for Human-Virtual Agents Interaction in Forum Theatre Inspired Training. Proceedings of the 16th International Workshop on Spoken Dialogue System Technology.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{otofa2026mixed,
  title = {{Mixed-Initiative Dialogue Management for Human-Virtual Agents Interaction in Forum Theatre Inspired Training}},
  author = {Samuel Otofa, Yacine Zerenini, Frederic Bechet, Benoit Favre, Jean-Marie Pergandi and Magalie Ochs},
  booktitle = {Proceedings of the 16th International Workshop on Spoken Dialogue System Technology},
  year = {2026},
  url = {https://aclanthology.org/2026.iwsds-1.11/}
}

References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brandon Amos, Bartosz Ludwiczuk, Mahadev Satyanarayanan, et al. 2016. Openface: A general-purpose face recognition library with mobile applications. CMU School of Computer Science, 6(2):20. find
  2. Keith Anderson, Elisabeth André, Tobias Baur, Sara Bernardini, Mathieu Chollet, Evi Chryssafidou, Ionut Damian, Cathy Ennis, Arjan Egges, Patrick Gebhard, et al. 2013. The tardis framework: intelligent virtual agents for social coaching in job interviews. In International Conference on Advances in C… find
  3. Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen. 2023. NeMo guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Dem… find
  4. Seongbum Seo, Sangbong Yoo, and Yun Jang. 2025. A prompt chaining framework for long-term recall in llm-powered intelligent assistant. In Proceedings of the 30th International Conference on Intelligent User Interfaces, pages 89–105. find
  5. Augusto Boal. 1972. Catégories du théâtre populaire. Travail théâtral, 6:3–26. find
  6. Merijn Bruijnes, Jeroen Linssen, and Dirk Heylen. 2019. Special issue editorial: Virtual agents for social skills training. find
  7. Leonardo Campillos-Llanos, D. Bouamor, Éric Bilinski, Anne-Laure Ligozat, Pierre Zweigenbaum, and S. Rosset. 2015. Description of the patientgenesys dialogue system. In SIGDIAL Conference. find
  8. Brigitte Grésy. 2009. Petit traité contre le sexisme ordinaire. Albin Michel. find
  9. Brigitte Grésy. 2015. Le sexisme dans le monde du travail, entre déni et réalité. Technical report, Rapport du Conseil supérieur de l’égalité professionnelle entre les femmes et les hommes solidaire. find
  10. Lynne Hall, Susan J Jones, Ruth Aylett, Elisabeth Andre, Ana Paiva, Gert Jan Hofstede, Arvid Kappas, Yukiko Nakano, and Toyoaki Nishida. 2011. Fostering empathic behaviour in children and young people: interaction with intelligent characters embodying culturally specific behaviour in virtual world s… find
  11. Gibbeum Lee, Volker Hartmann, Jongho Park, Dimitris Papailiopoulos, and Kangwook Lee. 2023. Prompted LLMs as chatbot modules for long open-domain conversation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4536–4554, Toronto, Canada. Association for Computational Ling… find
  12. Magalie Ochs, Jean-Marie Pergandi, Alain Ghio, Carine André, Patrick Sainton, Emmanuel Ayad, Auriane Boudin, and Roxane Bertrand. 2023. A forum theater corpus for discrimination awareness. Frontiers in Computer Science, 5:1081586. find
  13. D. Pautler, Vikram Ramanarayanan, Kirby Cofino, P. Lange, and D. Suendermann-Oeft. 2018. Leveraging multimodal dialog technology for the design of automated and interactive student agents for teacher training. In SIGDIAL Conference. find
  14. Alexander Prange, Margarita Chikobava, P. Poller, Michael Barz, and Daniel Sonntag. 2017. A multimodal dialogue system for medical decision support inside virtual reality. In SIGDIAL Conference. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/otofa-2026-mixed-initiative-dialogue-management-human/