📄 A Corpus for Personalized Dialogue Breakdown Repair in Japanese Open-Domain Conversations

Summary

LLM-based chat systems still produce conversational breakdowns, such as hallucinated facts, ignored questions or abrupt topic changes. Yet there has been no resource showing how a system should recover from them, or how the best recovery might depend on the user. Existing resources either avoid breakdowns entirely (Hazumi) or label them without repair responses or user profiles (the breakdown detection shared-task data).

The authors wrote 140 short Japanese dialogue snippets of two to four turns. Each ends in a system error drawn from ten types in the taxonomy of Higashinaka et al. (2022). Native-speaker crowd workers did three things for each snippet:

• rated how broken it felt,
• wrote their own reply as the user,
• wrote the repair they would expect from the system.

Workers also completed standard personality and social-skill questionnaires. Dialogue act tags were then assigned to both the user replies and the expected repairs. The result is a public corpus of 3,990 breakdown–repair examples linked to respondent traits. It supports research on repair strategies that adapt to individual users, and the paper includes an initial analysis.

Key points

• Scenario design: 10 breakdown types × 14 examples = 140 scenarios. Types include Wrong information, Ignore question, Topic transition error, Self-contradiction, Repetition and Lack of common sense. Each scenario states the intended cause of the error.
• Collection: 57 crowd workers each answered one of eight 70-scenario sets (7 per type), giving 3,990 examples. Use of generative AI was disabled in the crowdsourcing settings.
• Severity rating: each example has a breakdown severity score on a 1–10 scale. This lets researchers filter out cases that respondents did not perceive as breakdowns.
• Respondent traits: gender and age group, Big Five personality measured with TIPI-J, communicative ability measured with ENDCORE, dispositional shyness, and social skill levels measured with KiSS-18.
• Dialogue act tags:
  • User replies: Breakdown Indication, Confirmation, Continuation or Other.
  • System repairs: No Repair, Confirmation, Information Addition, Social Repair or Other.
  • Few-shot LLM tagging prompts are given in the appendix.
  • The corpus is available at github.com/kzy-tbkr/Dialogue-Breakdown-Repair-Corpus.