Elamur Research⌕
☰
home / papers / 2023 / DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

Summary

Implicit Discourse Relation Recognition (IDRR) asks a model to identify how two text spans (arguments) relate when no explicit connective such as "however" marks the relation. PDTB-style sense labels form a tree. Most earlier systems still treat the task as flat classification or rely only on connective prediction. Methods that do encode the label tree, such as LDSGM with graph convolutions, change the representations of the pre-trained model. The authors propose DiscoPrompt, which recasts IDRR as selecting a complete path through the hierarchy: a top-level sense, then a second-level sense, then a representative connective. The model uses a T5 backbone. Its template combines discrete, masked and soft tokens, and it places a textual version of the entire label tree before the argument pair. This lets the model score all candidate paths jointly. Experiments on PDTB 2.0 and CoNLL-2016 report new state-of-the-art results. This suggests that prompting with the label hierarchy can add structural knowledge while leaving the pre-trained representations intact.

Key points

  • The label set for PDTB 2.0 has 4 top-level relations, 11 second-level subtypes and 11 connectives, one per subtype. This yields 11 candidate paths, such as "Comparison -> Contrast -> however".
  • Connectives were chosen using Naive Bayes priors Pr(subtype | connective) estimated from explicit relation data. Adding more than one connective per subtype gave no clear benefit.
  • The method also transfers to other settings:
    • it works as a template for ordinary T5-large fine-tuning ("T5-adapt"), where it improved all metrics on CoNLL16;
    • on explicit discourse relation recognition, it slightly beats previous SOTA on top-level senses;
    • the authors also explore adapting it to ChatGPT.
  • The authors acknowledge several limitations:
    • weak scores on rare senses such as Comp.Concession and Expa.List;
    • no valid result for Cont.Pragmatic Cause;
    • gold connectives are excluded from the tree because they would create over 100 paths.
  • The code is released at github.com/HKUST-KnowComp/DiscoPrompt. The data is English news text (WSJ and newswire).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

29 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 21 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-acl.4 · arXiv 2305.03973

@inproceedings{chan2023discoprompt,
  title = {{DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition}},
  author = {Chan, Chunkit and Liu, Xin and Cheng, Jiayang and Li, Zihan and Song, Yangqiu and Wong, Ginny and See, Simon},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.4},
  eprint = {2305.03973},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.4}
}

References

52 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jonathan Gordon and Benjamin Van Durme. 2013. Reporting bias and knowledge acquisition. In AKBC, pages 25–30. find
  2. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. In COLING, pages 571–583. find our card
  3. Fengyu Guo, Ruifang He, Jianwu Dang, and Jian Wang. 2020. Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition. In AAAI, pages 7822–7829. find
  4. Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, and Yejin Choi. 2018. Discourse-aware neural rewards for coherent text generation. In NAACL-HLT, pages 173–184. find
  5. Ruifang He, Jian Wang, Fengyu Guo, and Yugui Han. 2020. Transs-driven joint learning architecture for implicit discourse relation recognition. In ACL, pages 139–148. find our card
  6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  7. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Trans. Assoc. Comput. Linguistics, 3:329–344. find our card
  8. Congcong Jiang, Tieyun Qian, and Bing Liu. 2022a. Knowledge distillation for discourse relation analysis. In WWW, pages 210–214. find
  9. Chunkit Chan, Jiayang Cheng, Weiqi Wang, Yuxin Jiang, Tianqing Fang, Xin Liu, and Yangqiu Song. 2023. Chatgpt evaluation on sentence level relations: A focus on temporal, causal, and discourse relations. CoRR, abs/2304.14827. find our card
  10. Feng Jiang, Yaxin Fan, Xiaomin Chu, Peifeng Li, and Qiaoming Zhu. 2021. Not just classification: Recognizing implicit discourse relation on joint modeling of classification and generation. In EMNLP, pages 2418–2431. find our card
  11. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  12. Yuxin Jiang, Chunkit Chan, Mingyang Chen, and Wei Wang. 2023. Lion: Adversarial distillation of closed-source large language model. CoRR, abs/2305.12870. — [arXiv:2305.12870](https://arxiv.org/abs/2305.12870) find
  13. Yuxin Jiang, Linhan Zhang, and Wei Wang. 2022b. Global and local hierarchy-aware contrastive framework for implicit discourse relation recognition. CoRR, abs/2211.13873. find our card
  14. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In NAACL-HLT, pages 141–151. find our card
  15. Najoung Kim, Song Feng, R. Chulaka Gunasekara, and Luis A. Lastras. 2020. Implicit discourse relation classification: We need to talk about evaluation. In ACL, pages 5404–5414. find our card
  16. Zeyu Dai and Ruihong Huang. 2019. A regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In EMNLP, pages 2974–2985. find our card
  17. Yudai Kishimoto, Yugo Murawaki, and Sadao Kurohashi. 2020. Adapting BERT to implicit discourse relation classification with a focus on discourse connectives. In LREC, pages 1152–1158. find our card
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186. find
  19. Murathan Kurfali and Robert Östling. 2021. Let’s be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction. CoRR, abs/2106.03192. find our card
  20. Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, and Maosong Sun. 2022. Openprompt: An open-source framework for prompt-learning. In ACL, pages 105–113. find
  21. Zujun Dou, Yu Hong, Yu Sun, and Guodong Zhou. 2021. CVAE-based re-anchoring for implicit discourse relation classification. In Findings of EMNLP, pages 1275–1283. find our card
  22. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In EMNLP, pages 1299–1308. find our card
  23. Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In ACL/IJCNLP, pages 3816–3830. find our card
  24. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In EMNLP, pages 3045–3059. find
  25. Jiaqi Li, Ming Liu, Bing Qin, and Ting Liu. 2022. A survey of discourse parsing. Frontiers Comput. Sci., 16(5):165329. find
  26. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In ACL, pages 13–16. find our card
  27. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In ACL/IJCNLP, pages 4582–4597. find
  28. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K. Joshi, and Bonnie L. Webber. 2008. The penn discourse treebank 2.0. In LREC. find our card
  29. Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. 2019. Kagnet: Knowledge-aware graph networks for commonsense reasoning. In EMNLP-IJCNLP, pages 2829–2839. find
  30. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016a. Implicit discourse relation recognition with context-aware character-enhanced embeddings. In COLING, pages 1914–1924. find our card
  31. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In EMNLP, pages 343–351. find our card
  32. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016b. A stacking gated neural architecture for implicit discourse relation classification. In EMNLP, pages 2263–2270. find our card
  33. Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022. Generated knowledge prompting for commonsense reasoning. In ACL, pages 3154– 3169. find
  34. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric P. Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In ACL, pages 1006–1017. find our card
  35. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021a. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. CoRR, abs/2107.13586. find our card
  36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67. find
  37. Xin Liu, Jiefu Ou, Yangqiu Song, and Xin Jiang. 2020. On the importance of word and sentence representation learning in implicit discourse relation classification. In IJCAI, pages 3830–3836. find
  38. Attapol Rutherford and Nianwen Xue. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In NAACL-HLT. find our card
  39. Xin Liu, Jiefu Ou, Yangqiu Song, and Xin Jiang. 2021b. Exploring discourse structures for argument impact classification. In ACL/IJCNLP, pages 3958–3969. find
  40. Attapol Rutherford and Nianwen Xue. 2016. Robust non-explicit neural discourse parser in english and chinese. In SIGNLL, pages 55–59. find our card
  41. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In EMNLP, pages 1224–1233. find our card
  42. Teven Le Scao and Alexander M. Rush. 2021. How many data points is a prompt worth? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 2627–2636. Association… find our card
  43. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In AAAI, pages 2750– 2756. find
  44. Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In EACL, pages 255–269. find our card
  45. Wanqiu Long and Bonnie Webber. 2022. Facilitating contrastive learning of discourse relational senses by exploiting the hierarchy of sense relations. In EMNLP, page 10704–10716. find our card
  46. Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In ICML, pages 4603–4611. find
  47. Linh The Nguyen, Ngo Van Linh, Khoat Than, and Thien Huu Nguyen. 2019. Employing the correspondence of relations and connectives to identify implicit discourse relations via label embeddings. In ACL, pages 4201–4207. find our card
  48. OpenAI. 2023. GPT-4 technical report. abs/2303.08774. find our card
  49. Wei Shi and Vera Demberg. 2019a. Learning to explic-itate connectives with seq2seq network for implicit discourse relation classification. In IWCS, pages 188– 199. find
  50. CoRR, Wei Shi and Vera Demberg. 2019b. Next sentence prediction helps implicit discourse relation classification within and across domains. In EMNLP-IJCNLP, pages 5789–5795. find our card
  51. TB OpenAI. 2022. Chatgpt: Optimizing language models for dialogue. OpenAI. find
  52. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/ stanford\_alpaca. ↗

↑ back to top