Elamur Research⌕
☰
home / papers / 2026 / Analysis of Child-Caregiver Interactions for Developing a Caregiver Spoken Dialogue System

Analysis of Child-Caregiver Interactions for Developing a Caregiver Spoken Dialogue System

Summary

NannyME is a Japanese remote childcare service in which human caregivers talk with children online. Staffing it is costly, and demand for childcare support keeps growing, so the authors want to partially automate it with a caregiver spoken dialogue system. As a first step, they study how these conversations actually unfold. They recorded sessions through the service. Each session has an introduction, three rounds of a "three-hint" quiz each followed by related free talk, and a closing. The result is the Remote Childcare Dialogue Data (RCDD).

They compared RCDD with parent–child speech from CEJC-Child and CHILDES on turn timing, backchannel use, and the communicative intent of utterances. Intent labels were assigned automatically by an LLM. Children and caregivers rarely talked over each other, unlike adults. Backchannel types differed by session part, and intent patterns shifted between parts, with caregivers steering the exchange. The authors conclude that a turn-based design may be enough instead of a full-duplex one. They also argue that dialogue control should be tailored to each part, possibly with a separate model per part.

Key points

  • Data and comparison corpora: RCDD was collected via NannyME and compared with CEJC-Child and CHILDES. The intent comparison used only CHILDES because of terms-of-use limits.
  • Turn-taking measures: Following Nguyen et al. (2023), the study measured IPU, Pause, Gap and Overlap as cumulative durations per minute. IPUs are speech segments separated by silences longer than 0.2 s. Child–caregiver talk showed less overlap than adult dialogue.
  • Backchannels: Backchannels were counted with regular expressions using the R/E/L/A reactive-token categories of Den et al. (2011). Responsive tokens were split into R1/R2/R3 by number of repetitions. The quiz part relied heavily on expressive interjections.
  • Utterance intentions: The 12 INCA-A categories were extended to 19 by separating initiating moves from responses. Labels were produced by an OpenAI model (gpt-5-nano) given up to 50 utterances of history. A check against 50 manually labeled utterances gave Cohen's kappa 0.71.
  • Patterns by session part: Questions dominated the introduction and free conversation, statements dominated the quiz, and markings and evaluations were frequent in the closing. Free talk often followed a question–response–follow-up structure, while the quiz formed a hint–answer–hint cycle.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Dialogue systems

Cite

Cite as Sanae Yamashita, Shota Mochizuki, Yuko Kuma, Ray Sakai, Ayaka Sasaki and Ryuichiro Higashinaka. 2026. Analysis of Child-Caregiver Interactions for Developing a Caregiver Spoken Dialogue System. Proceedings of the 16th International Workshop on Spoken Dialogue System Technology.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{yamashita2026analysis,
  title = {{Analysis of Child-Caregiver Interactions for Developing a Caregiver Spoken Dialogue System}},
  author = {Sanae Yamashita, Shota Mochizuki, Yuko Kuma, Ray Sakai, Ayaka Sasaki and Ryuichiro Higashinaka},
  booktitle = {Proceedings of the 16th International Workshop on Spoken Dialogue System Technology},
  year = {2026},
  url = {https://aclanthology.org/2026.iwsds-1.36/}
}

References

31 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Federico Biagi, Cristina Iani, and Luigi Biagiotti. 2025. The use of the social robot NAO in medical settings: How to facilitate interactions between healthcare professionals and patients with autism spectrum disorder. Frontiers in Psychiatry, 16:1675098. find
  2. Kübra Bodur, Mitja Nikolaus, Fatima Kassim, Laurent Prévot, and Abdellah Fourtassi. 2021. ChiCo: A multimodal corpus for the study of child conversation. In Proceedings of the Companion Publication of the 2021 International Conference on Multimodal Interaction, pages 158–163. find
  3. Kübra Bodur, Mitja Nikolaus, Laurent Prévot, and Abdellah Fourtassi. 2023. Using video calls to study children’s conversational development: The case of backchannel signaling. Frontiers in Computer Science, 5:1088752. find
  4. Marisa Casillas, Susan C. Bobb, and Eve V. Clark. 2016. Turn-taking, timing, and planning in early language acquisition. Journal of Child Language, 43(6):1310– 1337. find
  5. Malcolm Coulthard. 2014. An introduction to discourse analysis. Routledge. find
  6. Mirjam de Haas, Paul Vogt, Rianne van den Berghe, Paul Leseman, Ora Oudgenoeg-Paz, Bram Willemsen, Jan de Wit, and Emiel Krahmer. 2022. Engagement in longitudinal child-robot language learning interactions: Disentangling robot and task engagement. International Journal of Child-Computer Interaction,… find
  7. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: A speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037. our card
  8. Yasuharu Den, Nao Yoshida, Katsuya Takanashi, and Hanae Koiso. 2011. Annotation of Japanese response tokens and preliminary analysis on their distribution in three-party conversations. In Proceedings of the 2011 International Conference on Speech Database and Assessments, pages 168–173. find
  9. Alireza Esfandbod, Zeynab Rokhi, Ali F Meghdari, Alireza Taheri, Minoo Alemi, and Mahdieh Karimi. 2023. Utilizing an emotional robot capable of lip-syncing in robot-assisted speech therapy sessions for children with language disorders. International journal of social robotics, 15(2):165–183. find
  10. Tanya M Gallagher and Holly K Craig. 1982. An investigation of overlap in children’s speech. Journal of Psycholinguistic Research, 11(1):63–75. find
  11. Dhia Elhak Goumri, Abhishek Agrawal, Mitja Nikolaus, Hong Duc Thang Vu, Kübra Bodur, Elias Emmar, Cassandre Armand, Chiara Mazzocconi, Shreejata Gupta, Laurent Prévot, Benoit Favre, Leonor Becerra-Bonache, and Abdellah Fourtassi. 2024. CHICA: A developmental corpus of child-caregiver’s face-to-face… find
  12. Viktória Horváth and Valéria Krepsz. 2023. Temporal characteristics of child-adult conversations: Utterances and turn-taking. Taikomoji kalbotyra, (19):3– 13. find
  13. Tatsuya Kawahara, Takashi Yamaguchi, Koji Inoue, Katsuya Takanashi, and Nigel G. Ward. 2016. Prediction and generation of backchannel form for attentive listening systems. In Proceedings of the 17th Annual Conference of the International Speech Communication Association, pages 2890–2894. find
  14. Min Kyong Kim, Stefania Druga, Shaghayegh Esmaeili, Julia Woodward, Alex Shaw, Ayushi Jain, Jaida Langham, Kristy Hollingshead, Silvia B Lovato, Erin Beneteau, Jaime Ruiz, Lisa Anthony, and Alexis Hiniker. 2022. Examining voice assistants in the context of children’s speech. International Journal of… find
  15. Hanae Koiso, Yuichi Ishimoto, Iseki Yuriko, Noriko Eguchi, Wakako Kashino, Yoshiko Kawabata, Mariko Tanaka, Yayoi Tanaka, and Ken’ya Nishikawa. 2025. Construction of the pilot version of the corpus of everyday Japanese conversation for child. In Proceedings of the Thirty-first Annual Meeting of the… find
  16. Jacqueline M Kory-Westlund and Cynthia Breazeal. 2019. A long-term study of young children’s rapport, social emulation, and language learning with a peer-like robot playmate in preschool. Frontiers in Robotics and AI, 6:81. find
  17. Jing Liu and Abdellah Fourtassi. 2024. Benchmarking LLMs for mimicking child-caregiver language in interaction. arXiv preprint arXiv:2412.09318. find
  18. Jing Liu, Mitja Nikolaus, Kübra Bodur, and Abdellah Fourtassi. 2022. Predicting backchannel signaling in child-caregiver multimodal conversations. In Proceedings of the Companion publication of the 2022 international conference on multimodal interaction, pages 196–200. find
  19. Brian Macwhinney. 2000. The CHILDES Project: Tools for Analyzing Talk. Lawrence Erlbaum Associates. find
  20. Sayo Miyata. 2012. Japanese CHILDES: The 2012 CHILDES manual for Japanese. http://www2.aasa.ac.jp/people/smiyata/ CHILDESmanual/chapter01.html (in Japanese). ↗
  21. Jack Mostow, Greg Aist, Paul Burkhead, Albert Corbett, Andrew Cuneo, Susan Eitelman, Cathy Huang, Brian Junker, Mary Beth Sklar, and Brian Tobin. 2003. Evaluation of an automated reading tutor that listens: Comparison to human tutoring and classroom instruction. Journal of Educational Computing Rese… find
  22. Ying Xu, Joseph Aubele, Valery Vigil, Andres S. Bustamante, Young-Suk Kim, and Mark Warschauer. 2022. Dialogue with a conversational agent promotes children’s story comprehension via enhancing engagement. Child Development, 93:307–611. find
  23. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, and Emmanuel Dupoux. 2023. Generative spoken dialogue language modeling. Transactions of the Association for Computational Linguistics, 11:250–266. find
  24. Mitja Nikolaus, Eliot Maes, Jeremy Auguste, Laurent Prévot, and Abdellah Fourtassi. 2022. Large-scale study of speech acts’ development in early childhood. Language Development Research, 2(1):268–304. find
  25. Anat Ninio, Catherine E. Snow, Barbara A. Pan, and Pamela R. Rollins. 1994. Classifying communicative acts in children’s interactions. Journal of Communication Disorders, 27(2):157–187. find
  26. Atsumoto Ohashi, Shinya Iizuka, Jingjing Jiang, and Ryuichiro Higashinaka. 2025. Towards a Japanese full-duplex spoken dialogue system. In Proceedings of the 26th Interspeech Conference, pages 1783– 1787. find
  27. Nazerke Rakhymbayeva, Aida Amirova, and Anara Sandygulova. 2021. A long-term engagement with a social robot for autism therapy. Frontiers in robotics and AI, 8:669972. find
  28. Yoko Saikachi, Kazuki Watanabe, Takayuki Konishi, Naoko Ito, Ai Kanato, Yosuke Igarashi, Koki Miyazawa, Ken’ya Nishikawa, and Reiko Mazuka. 2013. Riken Japanese mother infant conversation corpus (R-JMICC) – compilation and recent findings of Japanese-specific prosodic and segmental characteristics i… find
  29. Emanuel A. Schegloff. 2007. Sequence Organization in Interaction: A Primer in Conversation Analysis. Cambridge University Press. find
  30. Emanuel A. Schegloff and Harvey Sacks. 1973. Opening up closings. Semiotica, 8(4):289–327. find
  31. Wayne Ward, Ronald Cole, Daniel Bolaños, Cindy Buchenroth-Martin, Edward Svirsky, Sarel Van Vuuren, Timothy Weston, Jing Zheng, and Lee Becker. 2011. My science tutor: a conversational multimedia virtual tutor for elementary school science. ACM Transactions on Speech and Language Processing, 7(4). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yamashita-2026-analysis-child-caregiver-interactions-developing/