Elamur Research⌕
☰
home / papers / 2023 / DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning

DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning

Paper · 2023 · Hengli Li, Song-Chun Zhu and Zilong Zheng

Summary

Machines often miss what a speaker means beyond the literal words. Earlier benchmarks also split figurative language into separate tasks such as metaphor, idiom or sarcasm. DiPlomat instead offers one umbrella benchmark for implied meaning in real two-party conversation.

The authors started from the Interview corpus of NPR radio transcripts. They filtered candidates automatically, using idiom string matching with the EPIE list and a RoBERTa hyperbole detector trained on Hypo-XL. AMT workers then marked turns whose intended sense differs from the literal one, adding a confidence score and a short rationale, and a human refinement pass followed.

The resulting benchmark holds 4,177 multi-turn dialogues and supports a pair of tasks, Conversational Question Answering (CQA) and Pragmatic Identification and Reasoning (PIR). Strong baselines stay below 0.70 accuracy on PIR and below 0.50 on CQA. In a zero-shot entailment probe, only ChatGPT, which saw the whole dialogue, clearly beats chance.

The work shows that context modelling and real reasoning, rather than pattern memorization, are the main obstacle for socially competent dialogue agents.

Key points

  • Source data: the two-party subset (23,714 dialogues) of the Interview dataset, built from about 20 years of NPR radio programs. Curation had three steps: automatic selection, fine-grained AMT annotation and human refinement.
  • Size: 4,177 dialogues with a 48,900-word vocabulary, 6,494 human-written answers with about 20,000 distinct words, and 19,482 CQA questions with human-written answers.
  • Annotation rules: workers flagged every turn where literal and intended meaning diverge, gave a confidence score and a reason of at most 8 words. Only workers from English-speaking countries with at least 1,300 approved tasks and a 98% approval rate could take part.
  • Results: on CQA the best model, ChatGPT, reaches 40.6%, and the authors note possible training-data exposure. On zero-shot NLI, DeBERTa-Large scores 44.3 and T5-XXL 45.3, both given only the turn. ChatGPT scores 85.7 with the full dialogue and drops to 63.8 with "Let's think step by step".
  • On PIR, models do well when context and the pragmatic turn are given and only the reason is asked (CP → R), but poorly when they must find the turn themselves (C → PR). The authors read this gap as memorization rather than understanding.

Interesting ideas

  • I1. Pair a cheap automatic prefilter (idiom lists, a hyperbole classifier) with human labels that each carry a confidence score and a very short free-text rationale, so subjective judgments arrive with graded certainty and a compact justification.
  • I2. Split one capability into "explain a given pragmatic turn" and "find the turn and explain it", then read a large gap between the two as a sign of memorized patterns rather than understanding.
  • I3. Measure how much context matters by blinding a system to the surrounding dialogue and showing it only the target turn, since the same sentence can be praise or sarcasm depending on the situation.
  • I4. Keep annotator disagreement as a range of acceptable readings instead of forcing one gold interpretation, because pragmatic meaning is inherently subjective and can even be read in opposite ways.
  • I5. Turn human-written implied meanings into hypotheses for a positives-only entailment check, giving a cheap zero-shot probe without building negative examples.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the implicature/irony detector in prag could emit a flag together with a confidence and a few-word rationale tied to the frame that triggered it, keeping each decision explainable.
  • I2 (learn): evaluation could pair each explain-only test with a locate-and-explain version, using the gap as a diagnostic alongside negative controls.
  • I3 (frames): run the irony and speech-act stage both with and without the preceding discourse frames, and use the contrast to show which pragmatic labels truly depend on context.
  • I4 (learn): when witness parsers or LLM teachers disagree on an implied meaning, store the competing readings with weights instead of discarding the item as noisy silver data.
  • I5 (answer): the reader could test candidate implied meanings against the world of the text as entailment hypotheses, so its answers about what a character "really meant" become checkable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksDialogue systems · 624 works

Cite

Cite as Hengli Li, Song-Chun Zhu and Zilong Zheng. 2023. DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning. arXiv:2306.09030.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2306.09030 · arXiv 2306.09030

@misc{li2023diplomat,
  title = {{DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning}},
  author = {Hengli Li, Song-Chun Zhu and Zilong Zheng},
  year = {2023},
  doi = {10.48550/arxiv.2306.09030},
  eprint = {2306.09030},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.09030}
}

References

69 references, 41 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jens Ambrasat, Christian von Scheve, Markus Conrad, Gesche Schauenburg, and Tobias Schröder. Consensus and stratification in the affective meaning of human sociality. Proceedings of the National Academy of Sciences, 111(22):8001–8006, 2014. 2 ↗
  2. Alan P Fiske. The four elementary forms of sociality: framework for a unified theory of social relations. Psychological review, 99(4):689, 1992. 2 ↗
  3. John A Bargh and Tanya L Chartrand. The unbearable automaticity of being. American psychologist, 54 (7):462, 1999. 2 ↗
  4. Edward Finegan. Language: Its structure and use. Cengage Learning, 2014. 2 find
  5. Meta Fundamental AI Research Diplomacy Team (FAIR)†, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378 (6624):1067–1074, 2022. 2 find
  6. Silke Anders, Roos de Jong, Christian Beck, John-Dylan Haynes, and Thomas Ethofer. A neural link between affective understanding and interpersonal attraction. Proceedings of the National Academy of Sciences, 113(16):E2248–E2257, 2016. 2 ↗
  7. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022. 2
  8. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020. 2 ↗
  9. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. 2 ↗
  10. Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, and Edward Grefenstette. Large language models are not zero-shot communicators, 2022. 2 find
  11. Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. We’re afraid language models aren’t modeling ambiguity, 2023. 2 — We're Afraid Language Models Aren't Modeling Ambiguity ↗
  12. Zilong Zheng, Shuwen Qiu, Lifeng Fan, Yixin Zhu, and Song-Chun Zhu. GRICE: A grammar-based dataset for recovering implicature and conversational rEasoning. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 2074–2085, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-acl.182. URL https://aclanthology.org/2021. findings-acl.182. 2, 3 ↗
  13. OpenAI. Gpt-4 technical report, 2023. 2, 3, 8
  14. Amelia Glaese, Nat McAleese, Maja Tr˛ebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375, 2022. 2 ↗
  15. Tuhin Chakrabarty, Xurui Zhang, Smaranda Muresan, and Nanyun Peng. Mermaid: Metaphor generation with symbolism and discriminative decoding. In Annual Meeting of the Association for Computational Linguistics (ACL), 2021. 2, 3 ↗
  16. Prateek Saxena and Soma Paul. Epie dataset: A corpus for possible idiomatic expressions. In Text, Speech, and Dialogue - 23rd International Conference, TSD 2020, Brno, Czech Republic, September 8-11, 2020, Proceedings, 2020. 2, 3, 4, 16 find
  17. Tosin P. Adewumi, Roshanak Vadoodi, Aparajita Tripathy, Konstantina Nikolaidou, Foteini Liwicki, and Marcus Liwicki. Potential idiomatic expression (pie)-english: Corpus for classes of idioms. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 2022. 2, 3
  18. Issa Annamoradnejad and Gohar Zoghi. Colbert: Using bert sentence embedding in parallel neural networks for computational humor, 2022. 2, 3, 16 find
  19. Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. Interview: Large-scale modeling of media dialog with discourse patterns and knowledge grounding. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8129–8141, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.653. URL https://aclanthology.org/2020.emnlp-main.653. 2, 4 ↗
  20. Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 285–294, Prague, Czech Republic, September 2015. Association for Computational Linguistics. doi: 10.18653/v1/W15-4640. URL https://aclanthology.org/W15-4640. 3 ↗
  21. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. Race: Large-scale reading comprehension dataset from examinations. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017. 3
  22. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018. 3 ↗
  23. Adina Williams, Nikita Nangia, and Samuel R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2018. 3, 9
  24. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In Annual Meeting of the Association for Computational Linguistics (ACL), 2018. 3 ↗
  25. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. Swag: A large-scale adversarial dataset for grounded commonsense inference. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018. 3 find
  26. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Cosmos qa: Machine reading comprehension with contextual commonsense reasoning. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019. 3 ↗
  27. Siva Reddy, Danqi Chen, and Christopher D. Manning. Coqa: A conversational question answering challenge. In Transactions of the Association for Computational Linguistics (TACL), 2019. 3 ↗
  28. Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. Dream: A challenge dataset and models for dialogue-based reading comprehension. In Transactions of the Association for Computational Linguistics (TACL), 2019. 3 find
  29. Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. Dialogue natural language inference. In Annual Meeting of the Association for Computational Linguistics (ACL), 2019. 3
  30. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019. 3 ↗
  31. Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang, and Ming Zhou. Mutual: A dataset for multi-turn dialogue reasoning. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020. 3, 4
  32. Paloma Jeretic, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. Are natural language inference models imppressive? learning implicature and presupposition. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020. 3
  33. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. Dialogpt: Large-scale generative pre-training for conversational response generation. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020. 3, 17 ↗
  34. Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill Dolan, and Jianfeng Gao. Godel: Large-scale pre-training for goal-directed dialog, 2022. 3 ↗
  35. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. Lamda: Language models for dialog applications, 2022. 3 ↗
  36. Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. Towards a human-like open-domain chatbot, 2020. 3 ↗
  37. Chulaka Gunasekara, Seokhwan Kim, Luis Fernando D’Haro, Abhinav Rastogi, Yun-Nung Chen, Mihail Eric, Behnam Hedayatnia, Karthik Gopalakrishnan, Yang Liu, Chao-Wei Huang, Dilek Hakkani-Tür, Jinchao Li, Qi Zhu, Lingxiao Luo, Lars Liden, Kaili Huang, Shahin Shayandeh, Runze Liang, Baolin Peng, Zheng Zhang, Swadheen Shukla, Minlie Huang, Jianfeng Gao, Shikib Mehri, Yulan Feng, Carla Gordon, Seyed Hossein Alavi, David Traum, Maxine Eskenazi, Ahmad Beirami, Eunjoon, Cho, Paul A. Crook, Ankita De, Alborz Geramifard, Satwik Kottur, Seungwhan Moon, Shivani Poddar, and Rajen Subba. Overview of the ninth dialog system technology challenge: Dstc9, 2020. 3 ↗
  38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 3
  39. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu. Palm 2 technical report, 2023. 3
  40. Yunxiang Zhang and Xiaojun Wan. MOVER: Mask, over-generate and rank for hyperbole generation. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 6018–6030, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.440. URL https://aclanthology.org/2022.naacl-main.440. 3, 16 ↗
  41. Orion Weller and Kevin Seppi. The rJokes dataset: a large scale humor collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6136–6141, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https://aclanthology.org/2020.lrec-1.753. 3 ↗
  42. Michael C. Frank and Noah D. Goodman. Predicting pragmatic reasoning in language games. Science, 336 (6084):998–998, 2012. doi: 10.1126/science.1218633. URL https://www.science.org/doi/abs/10.1126/science.1218633. 4 ↗
  43. Kevin Stowe, Prasetya Utama, and Iryna Gurevych. IMPLI: Investigating NLI models’ performance on figurative language. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 5375–5388, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.369. URL https://aclanthology.org/2022.acl-long.369. 4 ↗
  44. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. It’s not rocket science : Interpreting figurative language in narratives. In Annual Meeting of the Association for Computational Linguistics (ACL), 2022. 4 — It's not Rocket Science: Interpreting Figurative Language in Narratives ↗
  45. Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. Cicero: A dataset for contextualized commonsense inference in dialogues. In Annual Meeting of the Association for Computational Linguistics (ACL), 2022. 4 find
  46. Deepanway Ghosal, Pengfei Hong, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. Cider: Commonsense inference for dialogue explanation and reasoning. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2021. find
  47. Pei Zhou, Pegah Jandaghi, Bill Yuchen Lin, Justin Cho, Jay Pujara, and Xiang Ren. Probing commonsense explanation in dialogue response generation. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021. 4 find
  48. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. 2019. URL http://arxiv.org/abs/1907.11692. 4, 16, 17 ↗
  49. Yunxiang Zhang and Xiaojun Wan. Mover: Mask, over-generate and rank for hyperbole generation. In Annual Meeting of the Association for Computational Linguistics (ACL), 2022. 4 find
  50. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020. 5, 6, 16, 17 ↗
  51. Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. From recognition to cognition: Visual commonsense reasoning. In Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 6 find
  52. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 11 2019. URL http://arxiv.org/abs/1908.10084. 6 ↗
  53. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Annual Meeting of the Association for Computational Linguistics (ACL), 2019. 8
  54. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. In Journal of Machine Learning Research (JMLR), 2020. 8, 9 ↗
  55. Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. Unifiedqa: Crossing format boundaries with a single qa system. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020. 8
  56. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 483–498, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021. naacl-main.41. URL https://aclanthology.org/2021.naacl-main.41. 8 ↗
  57. Bill MacCartney and Christopher D. Manning. Modeling semantic containment and exclusion in natural language inference. In International Conference on Computational Linguistics (COLING), pages 521–528, Manchester, UK, August 2008. Coling 2008 Organizing Committee. URL https://aclanthology.org/C08-1066. 9 ↗
  58. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations (ICLR), 2021. 9, 17 ↗
  59. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 9
  60. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. TRUE: Re-evaluating factual consistency evaluation. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 3905–3920, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.287. URL https://aclanthology.org/2022.naacl-main.287. 9 ↗
  61. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), 2015. 9 ↗
  62. Anya Ji, Noriyuki Kojima, Noah Rush, Alane Suhr, Wai Keen Vong, Robert Hawkins, and Yoav Artzi. Abstract visual reasoning with tangram shapes. In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 582–601, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.emnlp-main.38.9 ↗
  63. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A lite BERT for self-supervised learning of language representations. 2020. 16
  64. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations (ICLR), 2020. 16, 17
  65. Rishabh Misra. News headlines dataset for sarcasm detection, 2022. 16 ↗
  66. Lennart Wachowiak, Dagmar Gromann, and Chao Xu. Drum up SUPPORT: Systematic analysis of image-schematic conceptual metaphors. In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 44–53, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.flp-1.7. 16 ↗
  67. Marti A. Hearst. Texttiling: Segmenting text into multi-paragraph subtopic passages. Comput. Linguist., 23(1):33–64, mar 1997. ISSN 0891-2017. 16 find
  68. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. 17
  69. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. 17 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-diplomat-dialogue-dataset-situated-pragmatic/