Elamur Research⌕
☰
home / papers / 2026 / Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting

Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting

Paper · 2026 · Angela Yifei Yuan, Christine De Kock and Christopher Leckie

Summary

The paper addresses forecasting whether an online discussion will turn hostile while it is still civil. Existing forecasters lean on lexical and semantic cues, so they need a lot of labelled data and transfer poorly across communities. The authors propose speech acts as a compact pragmatic layer that is less sensitive to topic vocabulary.

An LLM (gpt-oss-120b), prompted zero-shot with a curated inventory of 50 speech-act verbs plus "other", labels each sentence. The prompt asks for the intent a listener would perceive rather than the literal force. These labels serve only as an auxiliary training target, so no extraction runs at inference. Two hierarchical models (RoBERTa-Large utterance encoder plus a small Transformer over turns) combine derailment prediction with speech-act detection.

Across three corpora (WIKI, CMV and a new GITHUB set), the approach beats strong baselines, especially with little data and when moving between domains. The gain is largest when the target corpus is very different from the source. This matters for new or small platforms that lack annotation budgets.

Key points

  • Taxonomy and labelling. The taxonomy has 50 speech-act verbs plus "other", each with a short gloss and, where relevant, the speaker's mental state. Labels are assigned per sentence and then pooled into a multi-label set for each utterance.
  • Human check. On 85 sentences from 30 responses, the LLM labels reached Cohen's κ = 0.53 across all 51 classes. Collapsed into 5 broad classes (assertive, directive, commissive, expressive, other), agreement rose to κ = 0.88.
  • H_parallel. The derailment head sits beside the speech-act head. Training runs in two phases: speech-act detection alone first, then joint multi-task training.
  • H_sequential. A unidirectional GRU runs over the predicted speech-act probabilities, so the forecast relies mainly on the pragmatic signal.
  • New GITHUB corpus. It contains 202 derailed and 696 civil threads, sampled at roughly 4 civil threads per toxic one.
  • Corpus divergence. Distance between corpora was measured semantically (Wasserstein on all-mpnet-base-v2 embeddings) and lexically (CHI distance on SpaCy en_core_web_sm tokens), using 3000 responses per corpus. GITHUB is the most distant corpus, slightly closer to WIKI than to CMV.
  • Evaluation protocol. A new protocol averages scores over timesteps to judge turn-by-turn forecasting behaviour.

Interesting ideas

  • I1. Training-only auxiliary labels. Expensive labels (here, LLM-produced speech acts) supervise a side task during training only, shaping the shared representation while inference stays cheap.
  • I2. Perceived intent over literal force. Annotators are asked for the act a listener would hear, with explicit cues for irony, indirectness, covert aggression and maxim violations; a sarcastic "you're right" becomes disagreeing plus criticizing.
  • I3. Graded label granularity. Agreement is reported both on fine verbs and on Searle-style top categories, showing that coarse labels are reliable even when fine ones are noisy.
  • I4. Predicting transfer from measured distance. Semantic and lexical distances between corpora are computed first and then used to explain how much abstract features help on transfer.
  • I5. Forcing reliance on the abstract layer. A bottleneck makes the downstream decision depend only on the pragmatic sequence, not on raw text.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM-teacher speech-act or frame labels could act as a training-time auxiliary objective for gparse or the tagger, without adding an LLM call at runtime.
  • I2 (frames): crate prag could record both literal and perceived speech acts per clause, using these four cue types as explainable triggers for the irony and implicature rules.
  • I3 (frames): speech-act verbs could be stored under their Searle class, falling back to the class when fine-grained confidence or witness agreement is low.
  • I4 (learn): before applying the parser to a new genre, measuring its distance from the training treebanks could predict where silver data or repair rules are most needed.
  • I5 (answer): conversation-level readers could be tested on speech-act sequences alone, a negative-control-style check that the answer comes from the meaning layer rather than surface words.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of MelbourneMelbourne · Australia100 works · 22,618 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Speech act theory · 229 works

Cite

Cite as Angela Yifei Yuan, Christine De Kock and Christopher Leckie. 2026. Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting. arXiv:2608.25359.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2608.25359 · arXiv 2608.25359

@misc{yuan2026leveraging,
  title = {{Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting}},
  author = {Angela Yifei Yuan, Christine {De Kock} and Christopher Leckie},
  year = {2026},
  doi = {10.48550/arxiv.2608.25359},
  eprint = {2608.25359},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2608.25359}
}

References

43 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vibhor Agarwal, Anthony P. Young, Sagar Joglekar, and Nishanth Sastry. 2023. A graph-based context-aware model to understand online conversations. ACM Trans. Web, 18(1). ↗
  2. Ahmad Aljanaideh. 2025. Speech act patterns for improving generalizability of explainable politeness detection models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 18945– 18954, Vienna, Austria. Association for Computational Linguistics. ↗
  3. Enas Altarawneh, Ameeta Agrawal, Michael Jenkin, and Manos Papagelis. 2023. Conversation derailment forecasting with graph convolutional networks. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 160–169, Toronto, Canada. Association for Computational Linguistics. find
  4. John Langshaw Austin. 1975. How To Do Things With Words. Oxford University Press. ↗
  5. Cambridge University Press. 2026. Cambridge dictionary. find
  6. Jonathan P. Chang, Justin Cheng, and Cristian Danescu-Niculescu-Mizil. 2020. Don’t let me be misunderstood:comparing intentions and perceptions in online discussions. In Proceedings of The Web Conference 2020, WWW ’20, page 2066–2077, New York, NY, USA. Association for Computing Machinery. find
  7. Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019. Trouble on the horizon: Forecasting the derailment of online conversations as they develop. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4743–4754, Hong Kong, China. Association for Computational Linguistics. ↗
  8. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724– 1734, Doha, Qatar. Association for Computational Linguistics. ↗
  9. Dario Compagno, Elena V. Epure, Rébecca Deneckere-Lebas, and Camille Salinesi. 2018. Exploring digital conversation corpora with process mining. Corpus Pragmatics, 2(2):193–215. ↗
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. ↗
  11. Cliff Goddard and Anna Wierzbicka. 2013. Suggesting, apologizing, complimenting: English speech-act verbs. In Words and Meanings: Lexical Semantics Across Domains, Languages, and Cultures. Oxford University Press. find
  12. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6629–6640, Red Hook, NY, USA. Curran Associates Inc. ↗
  13. Yilun Hua, Nicholas Chernogor, Yuzhe Gu, Seoyeon Jeong, Miranda Luo, and Cristian Danescu-Niculescu-Mizil. 2024. How did we get here? summarizing conversation dynamics. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7452–7477, Mexico City, Mexico. Association for Computational Linguistics. ↗
  14. Mia Mohammad Imran, Robert Zita, Rebekah Copeland, Preetha Chatterjee, Rahat Rizvi Rahman, and Kostadin Damevski. 2025. Understanding and predicting derailment in toxic conversations on github. Preprint, arXiv:2503.02191. ↗
  15. Isnaeni Isnaeni, Mantasiah R., and Hasmawati Hasmawati. 2025. The function of assertive speech acts in the novel calabai by pepi al-bayqunie. KLASIKAL : JOURNAL OF EDUCATION, LANGUAGE TEACH-ING AND SCIENCE, 7(1):468–480. ↗
  16. Yova Kementchedjhieva and Anders Søgaard. 2021. Dynamic forecasting of conversation derailment. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7915– 7919, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  17. Idham Kholid, Agus Hidayat, Bela Triyani, and Ijlhe Stkip Pgri Bandar Lampung. 2024. An analysis of assertive and commissive speech acts in simon sinek’s speeches. IJLHE: International Journal of Language, Humanities, and Education, 7:305–314. ↗
  18. Adam Kilgarriff. 2001. Comparing corpora. International Journal of Corpus Linguistics, 6(1):97–133. find
  19. George Kour, Samuel Ackerman, Eitan Daniel Farchi, Orna Raz, Boaz Carmeli, and Ateret Anaby Tavor. 2022. Measuring the measuring tools: An automatic evaluation of semantic metrics for text corpora. In Proceedings of the Second Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 405–416, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  20. Hengli Li, Song-Chun Zhu, and Zilong Zheng. 2023. Diplomat: A dialogue dataset for situated pragmatic reasoning. In Advances in Neural Information Processing Systems, volume 36, pages 46856–46884. Curran Associates, Inc.
  21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. ↗
  22. Anais Ollagnier. 2024. CyberAgressionAdo-v2: Leveraging pragmatic-level information to decipher online hate in French multiparty chats. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4287–4298, Torino, Italia. ELRA and ICCL. find
  23. OpenAI. 2025. gpt-oss-120b & gpt-oss-20b model card. Preprint, arXiv:2508.10925. ↗
  24. Massimiliano Orsini and Dominique Brunato. 2025. Direct and indirect interpretations of speech acts: Evidence from human judgments and large language models. In Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), pages 837–848, Cagliari, Italy. CEUR Workshop Proceedings. find
  25. Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2020. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55(2):477–523. ↗
  26. Ines Reinig, Ines Rehbein, and Simone Paolo Ponzetto. 2024. How to do politics with words: Investigating speech acts in parliamentary debates. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8287– 8300, Torino, Italia. ELRA and ICCL. find
  27. Tal Ridnik, Emanuel Ben-Baruch, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. 2021. Asymmetric Loss For Multi-Label Classification . In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 82–91, Los Alamitos, CA, USA. IEEE Computer Society. find
  28. Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Towards sentiment and emotion aided multi-modal speech act classification in Twitter. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5727–5737, Online. Association for Computational Linguistics.
  29. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108. ↗
  30. John R. Searle. 1979. A Taxonomy of Illocutionary Acts, page 1–29. Cambridge University Press. find
  31. Cesare Spinoso-Di Piano, David Eric Austin, Pablo Piantanida, and Jackie CK Cheung. 2025. (RSA)²: A rhetorical-strategy-aware rational speech act framework for figurative language understanding. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20898–20938, Vienna, Austria. Association for Computational Linguistics. ↗
  32. Shivashankar Subramanian, Trevor Cohn, and Timothy Baldwin. 2019. Target based speech act classification in political campaign text. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 273–282, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  33. Son Quoc Tran, Tushaar Gangavarapu, Nicholas Chernogor, Jonathan P. Chang, and Cristian Danescu-Niculescu-Mizil. 2025. Conversations gone awry, but then? evaluating conversational forecasting models. Preprint, arXiv:2507.19470. ↗
  34. Daniel Vanderveken. 1990. Meaning and Speech Acts: Volume 1, Principles of Language Use, volume 1. Cambridge University Press, Cambridge. find
  35. Tom Viering and Marco Loog. 2023. The shape of learning curves: A review. IEEE Trans. Pattern Anal. Mach. Intell., 45(6):7799–7819. ↗
  36. Renxi Wang and Shi Feng. 2023a. Global-local modeling with prompt-based knowledge enhancement for emotion inference in conversation. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2120–2127, Dubrovnik, Croatia. Association for Computational Linguistics. find
  37. Renxi Wang and Shi Feng. 2023b. Global-local modeling with prompt-based knowledge enhancement for emotion inference in conversation. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2120–2127, Dubrovnik, Croatia. Association for Computational Linguistics. find
  38. Anna Wierzbicka. 1987. English speech act verbs: A Semantic Dictionary. Academic Press, Sydney. find
  39. Chien-Sheng Wu, Steven C.H. Hoi, Richard Socher, and Caiming Xiong. 2020. TOD-BERT: Pre-trained natural language understanding for task-oriented dialogue. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 917–929, Online. Association for Computational Linguistics. ↗
  40. Xingle Xu, Shi Feng, Yuan Cui, Yifei Zhang, and Daling Wang. 2025. Ac-eic: addressee-centered emotion inference in conversations. International Journal of Machine Learning and Cybernetics, 16(7-8):5113– 5130. ↗
  41. Jiaqing Yuan and Munindar P. Singh. 2023. Conversation modeling to predict derailment. Proceedings of the International AAAI Conference on Web and Social Media, 17(1):926–935. find
  42. Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. Conversations gone awry: Detecting early signs of conversational failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1350–1361, Melbourne, Australia. Association for Computational Linguistics. ↗
  43. Yunfan Zhang, Kathleen McKeown, and Smaranda Muresan. 2025. Forecasting conversation derailments through generation. In Proceedings of the 18th International Natural Language Generation Conference, pages 699–715, Hanoi, Vietnam. Association for Computational Linguistics. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yuan-2026-leveraging-speech-acts-low-data/