Elamur Research⌕
☰
home / papers / 2020 / Filtering conversations through dialogue acts labels for improving corpus-based convergence studies

Filtering conversations through dialogue acts labels for improving corpus-based convergence studies

Paper · 2020 · Simone Fuscone, Benoit Favre and Laurent Prévot

Summary

Studies of acoustic convergence, where speakers gradually sound more like their partner, give clean results in lab settings but weak or mixed ones on large natural corpora. The authors argue that part of the problem is that a conversation mixes very different activities, such as one person narrating while the other mostly gives back-channels. Comparing speakers across such mixed material blurs the effect. They therefore filter Switchboard turns by dialogue act type, using both the manual NXT annotation and a three-class automatic turn tagger built on a hierarchical BiLSTM. They then test whether the distance between partners in Energy, Pitch and Speech Rate shrinks from the first half of a conversation to the second. The test uses linear mixed models with speaker and topic random effects. Filtering to statements alone keeps the convergence effects found on the full corpus while using under 10% of the data, and it makes the Speech Rate result more significant. The authors conclude that automatic dialogue act tagging can serve as after-the-fact control over large uncontrolled corpora.

Key points

  • Data:
    • Switchboard has 2430 telephone conversations (about 260 h, 543 speakers who do not know each other).
    • 642 of these conversations carry dialogue act labels in the NXT version, reduced to 42 tags.
    • Three tags dominate: Statement 36%, Acknowledgment 19% and Opinion 13%.
  • Convergence measure:
    • Each conversation is split into two halves, and the absolute gap between the two speakers' mean values is computed in each half.
    • The model, fitted with lme4/lmerTest, is ∆V ~ interval + (1|topic) + (1|speaker A) + (1|speaker B).
  • Features:
    • Energy and F0 come from openSMILE. Energy is normalized by each speaker's mean in that conversation and computed only on frames with voicing probability above 0.65.
    • Speech Rate follows Cohen Priva et al.: actual duration divided by expected duration. It is used only for utterances longer than 5 tokens.
  • Turn tagger:
    • It assigns three classes: STA+OPI, BAC+AGR and OTH. Each turn receives the majority dialogue act among its segments, giving a 52/25/23% split.
    • It is a two-level hierarchical BiLSTM (after Auguste et al. 2018 and Yang et al. 2016).
    • Overall F1 is 81%, but OTH reaches only 0.55.
  • Results:
    • When an effect is significant, the distance always decreases.
    • The Statement subset (about 17 h) shows convergence in Energy and Speech Rate.
    • Opinion shows nothing, back-channels show only Energy, and F0 is never significant.
    • Automatically tagged data reproduce the effects found on the raw data and improve the significance of Speech Rate for STA+OPI.

Interesting ideas

  • I1. Use functional labels such as dialogue acts as a filter that makes a messy corpus homogeneous before measuring a subtle effect; a smaller but more uniform subset can reveal more than the whole.
  • I2. Merge fine-grained classes into a few coarse groups that matter for the task, here "main speaker" versus "listener" turns, to obtain a tagger that is reliable enough to act as a filter.
  • I3. Map labels from one segmentation onto another (dialogue act segments onto turns) by majority vote, giving a simple and explainable bridge between annotation layers.
  • I4. Ask explicitly whether the noise from automatic labels and the loss of data are outweighed by the gain in homogeneity, and check that automatic labels reproduce the findings from manual labels.
  • I5. Normalize each measure against the same speaker's own baseline, and drop units that are too short for the measure to be meaningful (Speech Rate on short utterances).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): When evaluating a parser or frame extractor, split the test material by speech act or sentence type, so that gains in one kind of text are not hidden by noise from other kinds.
  • I2 (frames): The speech-act layer in prag could expose a coarse tier (assertive/opinion vs. feedback/agreement vs. other) alongside the detailed labels, which downstream steps can trust more.
  • I3 (tree): The same majority-vote rule could transfer annotations between treebanks whose sentence or token boundaries disagree during UD dialect conversion.
  • I4 (learn): Silver data from witness parsers could be validated the same way, by confirming that conclusions drawn on gold subsets still hold on the larger silver set.
  • I5 (answer): Confidence-based scoring of readers could normalize per text and skip items too short to judge, instead of averaging over everything.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France104 works · 60,877 citations🎓Aix-Marseille UniversitéMarseille · France14 works · 38,957 citations🔬Laboratoire Parole et LangageAix-en-Provence · France5 works · 61 citations🎓Institut Universitaire de FranceParis · France4 works · 69 citations🔬Laboratoire d’Informatique et SystèmesMarseille · France3 works · 7 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Psychology · 798 works Experimental and Cognitive Psychology · 463 works

🧭 Topics 2 Dialogue systems · 401 works Speech act theory · 160 works

Cite

Cite as Simone Fuscone, Benoit Favre and Laurent Prévot. 2020. Filtering conversations through dialogue acts labels for improving corpus-based convergence studies. Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.sigdial-1.25

@inproceedings{fuscone2020filtering,
  title = {{Filtering conversations through dialogue acts labels for improving corpus-based convergence studies}},
  author = {Simone Fuscone, Benoit Favre and Laurent Prévot},
  booktitle = {Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2020},
  doi = {10.18653/v1/2020.sigdial-1.25},
  url = {https://doi.org/10.18653/v1/2020.sigdial-1.25}
}

References

26 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne H Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, et al. 1991. The hcrc map task corpus. Language and speech, 34(4):351–366. ↗
  2. Jeremy Auguste, Robin Perrotin, and Alexis Nasr. 2018. Annotation en actes de dialogue pour les conversations d’assistance en ligne. In Actes de la conférence Traitement Automatique de la Langue Naturelle, TALN 2018, page 577. find
  3. Douglas Bates, Martin Maechler, Ben Bolker, Steven Walker, et al. 2014. lme4: Linear mixed-effects models using eigen and s4. R package version, 1(7):1–23. find
  4. A Kuznetsova, P Bruun Brockhoff, and R Haubo Boje-sen Christensen. 2014. lmertest: tests for random and fixed effects for linear mixed effects models. See https://CRAN. R-project. org/package= lmerTest. ↗
  5. Rivka Levitan. 2014. Acoustic-prosodic entrainment in human-human and human-computer dialogue. Ph.D. thesis, Columbia University. ↗
  6. Rivka Levitan and Julia Hirschberg. 2011. Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions. In Proceedings of Interspeech 2011. ↗
  7. Michael Natale. 1975. Convergence of mean vocal intensity in dyadic communication as a function of social desirability. Journal of Personality and Social Psychology, 32(5):790. ↗
  8. Francesca Bonin, Céline De Looze, Sucheta Ghosh, Emer Gilmartin, Carl Vogel, Anna Polychroniou, Hugues Salamin, Alessandro Vinciarelli, and Nick Campbell. 2013. Investigating fine temporal dynamics of prosodic and lexical accommodation. In Proceedings of 14th Annual Conference of the International Speech Communication Association, Lyon, France. find
  9. Jennifer S Pardo. 2006. On phonetic convergence during conversational interaction. The Journal of the Acoustical Society of America, 119(4):2382–2393. ↗
  10. Sasha Calhoun, Jean Carletta, Jason M Brenier, Neil Mayo, Dan Jurafsky, Mark Steedman, and David Beaver. 2010. The nxt-format switchboard corpus: a rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language resources and evaluation, 44(4):387–419. ↗
  11. Uriel Cohen Priva and Chelsea Sanker. 2019. Limitations of difference-in-difference for measuring convergence. Laboratory Phonology: Journal of the Association for Laboratory Phonology, 10(1). ↗
  12. U Cohen Priva and C Sanker. 2018. Distinct behaviors in convergence across measures. In Proceedings of the 40th annual conference of the cognitive science society. Austin, TX. find
  13. Uriel Cohen Priva, Lee Edelist, and Emily Gleason. 2017. Converging to the baseline: Corpus evidence for convergence in speech rate to interlocutor’s baseline. The Journal of the Acoustical Society of America, 141(5):2989–2996. ↗
  14. Jens Edlund, Mattias Heldner, and Julia Hirschberg. 2009. Pause and gap length in face-to-face interaction. In Tenth Annual Conference of the International Speech Communication Association. find
  15. Florian Eyben and Björn Schuller. 2015. opensmile:): the munich open-source large-scale multimedia feature extractor. ACM SIGMultimedia Records, 6(4):4–13. find
  16. Simone Fuscone, Benoit Favre, and Laurent Prevot. 2018. Replicating speech rate convergence experiments on the switchboard corpus. In Workshop on Replicability and Reproducibility of Research Results in Science and Technology of Language. find
  17. H. Giles, N. Coupland, and J. Coupland. 1991. Accommodation theory: Communication, context, and consequence. In Contexts of accommodation: Developments in applied sociolinguistics, Studies in emotion and social interaction, pages 1–68. Cambridge University Press. ↗
  18. John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517–520. IEEE. find
  19. D Jurafsky, E Shriberg, and D Biasca. 1997. Switchboard dialog act corpus. International Computer Science Inst. Berkeley CA, Tech. Rep. find
  20. Martin J Pickering and Simon Garrod. 2004. Toward a mechanistic psychology of dialogue. Behavioral and brain sciences, 27(2):169–190. ↗
  21. Uwe D Reichel, Katalin Mády, and Jennifer Cole. 2018. Prosodic entrainment in dialog acts. arXiv preprint arXiv:1810.12646. ↗ our card
  22. Andreas Stolcke, Elizabeth Shriberg, Rebecca Bates, Noah Coccaro, Daniel Jurafsky, Rachel Martin, Marie Meteer, Klaus Ries, Paul Taylor, Carol Van Ess-Dykema, et al. 1998. Dialog act modeling for conversational speech. In AAAI Spring Symposium on Applying Machine Learning to Discourse Processing, pages 98–105. ↗
  23. Richard L. Street. 1984. Speech convergence and speech evaluation in fact-finding iinterviews. Human Communication Research, 11(2):139–169. find
  24. Khiet P Truong and Dirk Heylen. 2012. Measuring prosodic alignment in cooperative task-based conversations. In Thirteenth Annual Conference of the International Speech Communication Association. find
  25. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489. our card
  26. Victor H Yngve. 1970. On getting a word in edgewise. In Chicago Linguistics Society, 6th Meeting, pages 567–578. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fuscone-2020-filtering-conversations-through-dialogue-acts/