📄 Interoperability of Dialogue Corpora through ISO 24617-2-based Querying

Summary

Dialogue corpora are annotated with incompatible dialogue act schemes, which makes them hard to reuse together. Instead of building new corpora under the ISO 24617-2 standard or rewriting old annotations, the authors propose a third path: a query format based on DiAML, the standard's markup language. A user writes queries in ISO concepts, and the system resolves each query through a mapping to the native tags of the target corpus. To build the mappings, they compared the HCRC MapTask and AMI schemes with ISO 24617-2 using guidelines, examples and corpus data. They also related DAMSL-family tags to ISO categories. To check the mappings, they manually re-annotated a sample of dialogues with ISO tags. They then scored the retrieved results against this reference, using precision and recall for each ISO function. Most functions were retrieved with reasonable accuracy. Feedback elicitation in MapTask was the main exception, because recognising it needs wider dialogue context. Because the same query format works on two very different corpora, the approach appears portable to other dialogue resources.

Key points

• The two test corpora are HCRC MapTask (128 two-person instruction dialogues, one-dimensional scheme with 12 mutually exclusive tags) and AMI (100 hours of multiparty design meetings, with extra layers and relational tags such as POSitive, NEGative, PARTial and UNCertain).
• ISO 24617-2 is drawn from DIT++. It defines 9 dimensions (Task, Auto-/Allo-feedback, Own/Partner Communication Management, Time, Turn, Dialogue Structuring, Social Obligations), with 41 dimension-specific and 26 general-purpose functions.
• The reference data came from re-annotating 4 AMI dialogues (3,897 utterances) and 8 MapTask dialogues (1,728 utterances) with ISO tags.
• Set questions reached roughly 88% precision and 91–92% recall in both corpora. MapTask Feedback Elicitation dropped to 52.2% precision and 21.8% recall, because MapTask 'align', 'ready' and some query moves cannot be separated without wider context.
• Two querying options are contrasted:
  • converting corpora into DiAML and querying them with XPath/XQuery, which requires knowing each native scheme;
  • a scheme-independent DiAML query interface that hides corpus-specific formats.

Interesting ideas

• I1. Achieve interoperability at query time: keep each resource in its native annotation and translate a canonical query through a mapping, instead of converting the data itself.
• I2. Validate a tag mapping empirically: hand-annotate a small sample in the target scheme and report precision and recall for each label, so weak mappings show up one by one.
• I3. Allow several functional tags on one utterance across independent dimensions (task content, feedback, turn-taking, social obligations), rather than forcing a single label.
• I4. Express nuances like certainty or polarity as qualifiers on a base function (Inform with an Uncertain qualifier, for instance), not as separate atomic tags.
• I5. The failure analysis shows that some acts, such as feedback elicitation, cannot be identified from the utterance alone and need the preceding dialogue history.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): Treebanks in different UD dialects could be searched with one canonical UD query that is mapped onto each dialect, so no lossy bulk conversion is needed.
• I2 (learn): Every dialect converter or label bridge could carry per-label precision/recall from a small gold sample, which flags which converted labels are unsafe as silver data.
• I3 (frames): Speech-act analysis in prag could assign one act per dimension, e.g. an answer that also signals understanding, which keeps the explanation of multifunctional utterances transparent.
• I4 (frames): Speech-act frames could add certainty/sentiment qualifier slots to a small core act set, which keeps the inventory compact while staying expressive.
• I5 (frames): Speech-act classification could be allowed to consult the left context or discourse history explicitly, and could mark context-dependent labels as low-confidence when that history is missing.