Elamur Research⌕
☰
home / papers / 2010 / Automatic Extraction of Cue Phrases for Cross-Corpus Dialogue Act Classification

Automatic Extraction of Cue Phrases for Cross-Corpus Dialogue Act Classification

Paper · 2010 · Nick Webb and Michael Ferguson

Summary

The paper asks whether word n-grams that signal dialogue acts (DAs) can be mined automatically from one annotated corpus and still work on a different corpus. Webb and Ferguson take all n-grams of length 1–4 from training utterances. For each n-gram they compute its maximal predictivity, the highest conditional probability P(d|c) over DA labels, and keep candidates that pass a predictivity threshold and a minimum count. Instead of passing the cues to a learner, they classify directly: an utterance receives the DA of its most predictive cue, with ties broken by frequency. Utterances with no cue fall back to the most frequent label. Experiments use Switchboard (42 DAMSL-derived labels) and ICSI-MRDA with 10-fold cross-validation. Cues transferred between the two corpora keep a large share of within-corpus accuracy. Pruning the cue set down to a few hundred phrases shared by both corpora, mostly anchored at the start of the utterance, leaves accuracy unchanged. The result suggests that a small, portable core of lexical cues carries most of the signal for DA recognition and could support first-pass annotation of new corpora.

Key points

  • Cue selection uses two thresholds, maximal predictivity ≥ 30% and corpus count ≥ 2, applied to n-grams of length 1–4. The data comprise about 200,000 Switchboard utterances and about 105,000 ICSI-MRDA utterances.
  • Training on Switchboard cues and testing on ICSI-MRDA gives 47.07% accuracy. This is about 81% of the in-corpus result and 71% of the best published score (Ji and Bilmes, 2005).
  • In the reverse direction, ICSI-MRDA cues applied to Switchboard reach 70.78%, roughly 88% of the in-corpus Switchboard result.
  • Keeping only cues that match across both corpora leaves about 25k cues, and accuracy rises slightly to 72.34% (Switchboard) and 47.86% (ICSI-MRDA). A cue counts as matching only if the phrase, its position tag and its predicted DA all agree.
  • Further pruning does not hurt accuracy:
    • Dropping cues that predict the default <STATEMENT> label leaves 577 cues.
    • Keeping only cues with a <start> tag leaves 242 cues (72.52% / 47.30%).
    • Keeping only the 148 cues present in every fold gives 72.09% / 46.34%.

Interesting ideas

  • I1. Evaluate extracted features by using them as a bare classifier (argmax of P(d|c), ties broken by frequency), which gives fast feedback on their quality without a learner in the way.
  • I2. Treat "maximal predictivity plus a minimum count" as a transparent recipe for mining indicator phrases, where each decision traces back to one named n-gram.
  • I3. Keep only cues on which two independent corpora agree (same phrase, same position, same predicted label), which removes corpus-specific noise at no cost in accuracy.
  • I4. Drop any rule whose output equals the fallback default, since it cannot change a decision and only enlarges the model.
  • I5. Position anchoring matters: cues tied to the start of the utterance did most of the work, so position markers are worth encoding in n-gram features.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): candidate repair rules or lexical triggers could be scored this way first, so weak rules are filtered out before costlier training or evaluation.
  • I2 (frames): speech-act tagging in prag could use such a mined cue list as an explainable first layer, with each assigned act citing the phrase that triggered it.
  • I3 (learn): this mirrors witness agreement for silver data and could be applied to lexical rules induced from different treebanks or genres.
  • I4 (frames): a default-act fallback (e.g. assertion) would let prag store only cues for non-default acts, keeping the rule set small and easy to inspect.
  • I5 (frames): clause-initial tokens such as discourse markers or wh-words, located via the UD tree, could serve as primary speech-act evidence in prag.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: University at Albany, State University of New York.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Nick Webb and Michael Ferguson. 2010. Automatic Extraction of Cue Phrases for Cross-Corpus Dialogue Act Classification. Coling 2010: Posters.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{webb2010automatic,
  title = {{Automatic Extraction of Cue Phrases for Cross-Corpus Dialogue Act Classification}},
  author = {Nick Webb and Michael Ferguson},
  booktitle = {Coling 2010: Posters},
  year = {2010},
  url = {https://aclanthology.org/C10-2150/}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ang, J., Y. Liu, and E. Shriberg. 2005. Automatic Dialog Act Segmentation and Classification in Multi-party Meetings. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pages 1061–1064, Philadelphia. find
  2. Bunt, H. 1994. Context and Dialogue Control. THINK, 3:19–31. find
  3. Core, M., M. Ishizaki, J. Moore, and C. Nakatani. 1999. The Report of the Third Workshop of the Discourse Resource Initiative. Chiba University and Kazusa Academia Hall. find
  4. Grosz, B. and C. Sidner. 1986. Attention, Intentions, and the Structure of Discourse. Computational Linguistics, 19(3). ↗
  5. Hirschberg, J. and D. Litman. 1993. Empirical Studies on the Disambiguation of Cue Phrases. Computational Linguistics, 19(3):501–530.
  6. Ji, G. and J. Bilmes. 2005. Dialog Act Tagging Using Graphical Models. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, PA. find
  7. Ji, G. and J. Bilmes. 2006. Backoff Model Training using Partially Observed Data: Application to Dialog Act Tagging. In Proceedings of the Human Language Technology/ American chapter of the Association for Computational Linguistics(HLT/NAACL’06). find
  8. Jurafsky, D., R. Bates, N. Coccaro, R. Martin, M. Meteer, K. Ries, E. Shriberg, A. Stolcke, P. Taylor, and C. Van Ess-Dykema. 1997. Automatic Detection of Discourse Structure for Speech Recognition and Understanding. In Proceedings of the 1997 IEEE Workshop on Speech Recognition and Understanding, Santa Barbara. ↗
  9. Lendvai, P. and J. Geertzen. 2007. Token-Based Chunking of Turn-Internal Dialogue Act Sequences. In Proceedings of the 8th SIGDIAL Workshop on Discourse and Dialogue, pages 174–181, Antwerp, Belgium.
  10. Levin, L., C. Langley, A. Lavie, D. Gates, and D. Wallace. 2003. Domain Specific Speech Acts for Spoken Language Translation. In Proceedings of 4th SIGdial Workshop on Discourse and Dialogue. find
  11. Meteer, M. 1995. Dysfluency Annotation Stylebook for the Switchboard Corpus. Working paper, Linguistic Data Consortium. find
  12. Reithinger, N. and M. Klesen. 1997. Dialogue Act Classification Using Language Models. In Proceedings of EuroSpeech-97. find
  13. Rotaru, M. 2002. Dialog Act Tagging using Memory-Based Learning. Term project, University of Pittsburgh. find
  14. Samuel, K., S. Carberry, and K. Vijay-Shanker. 1999. Automatically Selecting Useful Phrases for Dialogue Act Tagging. In Proceedings of the Fourth Conference of the Pacific Association for Computational Linguistics, Waterloo, Ontario, Canada. ↗
  15. Shriberg, E., R. Dhillon, S. Bhagat, J. Ang, and H. Carvey. 2004. The ICSI Meeting Recorder Dialog Act (MRDA) Corpus. In Special Interest Group on Discourse and Dialogue (SIGdial), Boston, USA. ↗
  16. Stolcke, A., K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer. 2000. Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. In Computational Linguistics 26(3), 339–373. ↗
  17. Verbree, D., R. Rienks, and D. Heylen. 2006. Dialogue Act Tagging using Smart Feature Selection; Results on Multiple Corpora. find
  18. Spoken Language Technology Workshop, 2006. IEEE, pages 70–73. find
  19. Webb, N. and T. Liu. 2008. Investigating the Portability of Corpus-Derived Cue Phrases for Dialogue Act Classification. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008), Manchester, United Kingdom. find
  20. Webb, N., M. Hepple, and Y. Wilks. 2005a. Dialogue Act Classification Based on Intra-Utterance Features. In Proceedings of the AAAI Workshop on Spoken Language Understanding, at the Twentieth National Conference on Artificial Intelligence, Pittsburgh, PA. find
  21. Webb, N., M. Hepple, and Y. Wilks. 2005b. Empirical Determination of Thresholds for Optimal Dialogue Act Classification. In Proceedings of the Ninth Workshop on the Semantics and Pragmatics of Dialogue. find
  22. Webb, N., M. Hepple, and Y. Wilks. 2005c. Error Analysis of Dialogue Act Classification. In Proceedings of the 8th International Conference on Text, Speech and Dialogue, Carlsbad, Czech Republic. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/webb-2010-automatic-extraction-cue-phrases-cross/