Elamur Research⌕
☰
home / papers / 2026 / LLM-Based Zero-Shot Soft Labeling for Anticipating Disagreement in Negotiation Dialogues

LLM-Based Zero-Shot Soft Labeling for Anticipating Disagreement in Negotiation Dialogues

Paper · 2026 · Ken Watanabe and Katsuhide Fujita

Summary

Negotiation utterances often carry indirect or blended intentions. A question can work as an offer, and a seeming acceptance may only acknowledge a deadlock. Assigning each utterance a single dialogue-act tag therefore discards information. The authors propose ZeSTEM, a zero-shot technique that uses an LLM-based text embedding model to compare each utterance with every act in a dialogue-act inventory. The result is a graded similarity vector rather than a one-hot tag, and it requires no annotated data or training.

The soft labels are first checked against existing rule-based hard labels. Both label types are then used as inputs for FDANMD, which asks whether a negotiation will end in disagreement, judged from only part of the dialogue. Experiments on CraigslistBargain (CB), DealOrNoDeal (DN) and JobInterview (JI) show that soft labels generally yield higher ROC AUC. The gain is clearest early in the conversation, and soft labels hold up better when predictors are trained on several domains or tested on an unseen one. This suggests that cheap, domain-independent disagreement signals could support negotiation assistants and autonomous agents.

Key points

  • Agreement with rule-based hard labels reached a maximum HIT@3 of 0.87.
    • The 8B embedding model clearly outperformed the 0.6B one.
    • Results depended on label format, on whether the preceding utterance was added as context, and on the instruction text.
    • Some mismatches exposed errors in the rule-based labels themselves.
  • FDANMD was evaluated with ROC AUC at negotiation progress rates r = 0.25, 0.5 and 1.0. Seven training-data combinations of CB, DN and JI were compared, with significance tested at p < 0.05 and p < 0.01.
  • Soft labels helped most at r = 0.25. With hard labels, adding other-domain training data lowered performance at low progress rates, while soft-label models stayed stable.
  • JI is small and contains few disagreements. On JI, soft-label models trained on CB+DN (JI unseen) or on all three corpora consistently beat a model trained on JI alone.
  • The authors list three limitations:
    • Only agreement versus disagreement was predicted; outcomes such as final profit or satisfaction were not tested.
    • The act inventory is coarse. It does not separate initial offers from counter-offers, soft from hard refusals, or conditional acceptance from simple agreement.
    • The soft labels received no human evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tokyo University of Agriculture and TechnologyTokyo · Japan3 works · 24 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Ken Watanabe and Katsuhide Fujita. 2026. LLM-Based Zero-Shot Soft Labeling for Anticipating Disagreement in Negotiation Dialogues. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.90

@inproceedings{watanabe2026llm,
  title = {{LLM-Based Zero-Shot Soft Labeling for Anticipating Disagreement in Negotiation Dialogues}},
  author = {Ken Watanabe and Katsuhide Fujita},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.90},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.90}
}

References

28 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John L. Austin. 1962. How to Do Things with Words. Clarendon Press, Oxford [Eng.]. find
  2. Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou. 2024. How well can llms negotiate? negotiationarena platform and analysis. arXiv preprint arXiv:2402.05863. find
  3. Andrew P. Bradley. 1997. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7):1145–1159. find
  4. Kushal Chawla, Jaysa Ramirez, Rene Clever, Gale Lucas, Jonathan May, and Jonathan Gratch. 2021. CaSiNo: A corpus of campsite negotiation dialogues for automatic negotiation systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:… find
  5. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua…
  6. Angus Dempster, François Petitjean, and Geoffrey I. Webb. 2020. Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5):1454–1495. find
  7. Angus Dempster, Daniel F. Schmidt, and Geoffrey I. Webb. 2021. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 248–257, New York, NY, USA. Association for Com… find
  8. Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua. 2023. Plug-and-play policy planner for large language model powered dialogue agents. arXiv e-prints, pages arXiv–2311. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  10. Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, and Massimo Poesio. 2021. Beyond black & white: Leveraging annotator disagreement via soft-label multi-task learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Lin…
  11. He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333–2343, Brussels, Belgium. Association for Computational Linguistics.
  12. Mourad Heddaya, Solomon Dworkin, Chenhao Tan, Rob Voigt, and Alexander Zentefis. 2023. Language of bargaining. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13161– 13185, Toronto, Canada. Association for Computational Lingui… find
  13. Ryuichiro Higashinaka, Kotaro Funakoshi, Yuka Kobayashi, and Michimasa Inaba. 2016. The dialogue breakdown detection challenge: Task description, datasets, and evaluation metrics. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3146– 3150, P… find
  14. Arthur E. Hoerl and Robert W. Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67. find
  15. Franz Király, Markus Löning, Tony Bagnall, Matthew Middlehurst, Anirban Ray, Sajaysurya Ganesh, Martin Walter, George Oastler, Jason Lines, ViktorKaz, Benedikt Heidrich, Lukasz Mentel, Jigyasu, Sagar Mishra, chrisholder, Daniel Bartling, Armaghan Shakir, Leonidas Tsaprounis, RNKuhns, and 10 others.… find
  16. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  17. Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443–2453, Copenhagen, Denmark. Association for Computational Li…
  18. Markus Löning, Anthony Bagnall, Sajaysurya Ganesh, Viktor Kazakov, Jason Lines, and Franz J. Király. 2019. sktime: A unified interface for machine learning with time series. Preprint, arXiv:1909.07872. — [arXiv:1909.07872](https://arxiv.org/abs/1909.07872) find
  19. Johannes Mario Meissner, Napat Thumwanit, Saku Sugawara, and Akiko Aizawa. 2021. Embracing ambiguity: Shifting the training target of NLI models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language…
  20. Atsuki Yamaguchi, Kosui Iwasa, and Katsuhide Fujita. 2021. Dialogue act-based breakdown detection in negotiation dialogues. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 745–757, Online. Association for Computationa…
  21. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  22. Yiheng Zhou, He He, Alan W Black, and Yulia Tsvetkov. 2019. A dynamic strategy coach for effective negotiation. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 367–378, Stockholm, Sweden. Association for Computational Linguistics. find
  23. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:282… find
  24. Johannes Schneider, Steffi Haag, and Leona Chandra Kruse. 2023. Negotiating with llms: Prompt hacks, skill gaps, and reasoning deficits. arXiv e-prints, pages arXiv–2312. find
  25. John R. Searle. 1975. Indirect speech acts. In Peter Cole and Jerry L. Morgan, editors, Syntax and Semantics 3: Speech Acts, volume 3, pages 59–82. Academic Press, New York. find
  26. Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5635–5649, Florenc…
  27. Ken Watanabe and Katsuhide Fujita. 2025. Fine-tuning models for final disagreement anticipation in negotiation mid-dialogues. IEICE Transactions on Information and Systems, E108.D(3):286–294. find
  28. Ben Wu, Yue Li, Yida Mu, Carolina Scarton, Kalina Bontcheva, and Xingyi Song. 2023. Don’t waste a single annotation: improving single-label classifiers through soft labels. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5347–5355, Singapore. Association for Computati… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/watanabe-2026-llm-based-zero-shot-soft/