Elamur Research⌕
☰
home / papers / 2004 / Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies

Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies

Summary

The paper addresses automatic detection of agreement and disagreement between participants in multi-party meetings, which the authors see as a step toward meeting summarization. The work uses the ICSI Meeting corpus. Speech is split into pause-delimited spurts, and each spurt is labeled agreement, disagreement, backchannel, or other.

The method has two stages:

  1. Finding addressees. A maximum entropy ranker identifies adjacency pairs, which tell the system who is responding to whom. It ranks candidate speakers using structural, durational, and lexical cues drawn from both preceding and following context.
  2. Labeling spurts. A sequential classifier, structured as a Bayesian network, assigns the labels. Beyond local features, it encodes pragmatic dependencies on earlier labels, for instance whether a speaker who previously disagreed with someone tends to do so again.

On three-way classification, the system reaches 86.9% accuracy, compared with 82% for Hillard et al. (2003). Adding the label-dependency features contributes about 1.3 points of that gain. The results suggest that modeling conversational context and addressee structure improves pragmatic tagging in multi-party dialog.

Key points

  • Data: The ICSI Meeting corpus has 75 meetings averaging 6.5 participants. Seven meetings were annotated at the spurt level, giving 8135 spurts. The class split is 11.9% agreement, 6.8% disagreement, 23.2% backchannel, and 58.1% other. Inter-annotator kappa was 0.63.
  • Adjacency-pair identification: Simply choosing the most recent prior speaker is correct 79.8% of the time. The maxent ranker, trained with GIS in YASMET, reaches 89.38%. Adding Gaussian smoothing and feature selection raises this to 90.2%, and dialog-act tags bring it to 91.34%.
  • Forward-looking features: Dropping them lowers ranking accuracy to 86.99%.
  • Contextual dependencies: Four label-dependency features are modeled: the previous tag, the previous tag between the same interactants, reflexivity, and transitivity. Decoding uses beam search with a beam size of 100.
  • Four-way cross-validation: Accuracy is 84.07% with label dependencies versus 83.11% without them. Lexical features are the strongest of the local feature sets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

59 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 57 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1218955.1219040

@inproceedings{galley2004identifying,
  title = {{Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies}},
  author = {Galley, Michel and McKeown, Kathleen and Hirschberg, Julia and Shriberg, Elizabeth},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1219040},
  url = {https://doi.org/10.3115/1218955.1219040}
}

References

19 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Berger, S. Della Pietra, and V Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–72. find our card
  2. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological measurements, 20:37–46. find
  3. S. Cohen. 2002. A computerized scale for monitoring levels of agreement during a conversation. In Proc. of the 26th Penn Linguistics Colloquium. find
  4. M. Collins. 2000. Discriminative reranking for natural language parsing. In Proc. 17th International Conf. on Machine Learning, pages 175– 182. find our card
  5. J. N. Darroch and D. Ratcliff. 1972. Generalized iterative scaling for log-linear models. Annals of Mathematical Statistics, 43:1470–1480. find
  6. R. Dhillon, S. Bhagat, H. Carvey, and E. Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical Report TR-04-002, ICSI. find
  7. V. Hatzivassiloglou and K. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proc. of ACL. find
  8. D. Hillard, M. Ostendorf, and E Shriberg. 2003. Detection of agreement vs. disagreement in meetings: training with unlabeled data. In Proc. of HLT/NAACL. find
  9. J. Hirschberg and D. Litman. 1994. Empirical studies on the disambiguation of cue phrases. Computational Linguistics, 19(3):501–530. find
  10. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, A. Stolcke, and C. Wooters. 2003. The ICSI meeting corpus. In Proc. of ICASSP-03, Hong Kong. find
  11. D. Klein and C. D. Manning. 2002. Conditional structure versus conditional estimation in NLP models. Technical report. find
  12. S. Levinson. 1983. Pragmatics. Cambridge University Press. find
  13. A. McCallum, D. Freitag, and F. Pereira. 2000. Maximum entropy markov models for information extraction and segmentation. In Proc. of ICML. find
  14. A. Pomerantz. 1984. Agreeing and disagreeing with assessments: some features of preferred/dispreferred turn shapes. In J.M. Atkinson and J.C. Heritage, editors, Structures of Social Action, pages 57–101. find
  15. A. Ratnaparkhi. 1996. A maximum entropy part-of-speech tagger. In Proc. of EMNLP. find
  16. D. Ravichandran, E. Hovy, and F. J. Och. 2003. Statistical QA - classifier vs re-ranker: What’s the difference? In Proc. of the ACL Workshop on Multilingual Summarization and Question Answering. find
  17. E. A. Schegloff and H Sacks. 1973. Opening up closings. Semiotica, 7-4:289–327. find
  18. E. Shriberg, R. Dhillon, S. Bhagat, J. Ang, and H. Carvey. 2004. The ICSI meeting recorder dialog act (MRDA) corpus. In SIGdial Workshop on Discourse and Dialogue, pages 97–100. find our card
  19. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/galley-2004-identifying-agreement-disagreement-conversational-speech/