Elamur Research⌕
☰
home / papers / 2023 / TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition

TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition

Summary

The paper addresses Implicit Discourse Relation Recognition (IDRR): identifying the sense relation (e.g., Contingency, Comparison) between two arguments when no explicit connective is present. The earlier ConnPrompt model handled this as connective cloze prediction and combined three nearly identical templates by majority vote. TEPrompt replaces that ensemble with joint training of one main prompt task and two auxiliary ones:

  • DRR (Discourse Relation Recognition): the main connective-cloze task.
  • SSC (Sense Semantics Classification): predicts the relation label words directly from the PLM vocabulary.
  • ACP (Annotated Connective Prediction): predicts the connective that PDTB annotators inserted.

The three templates are concatenated into a single PLM input and share argument representations. During training, all three tasks are optimized together. At test time, only the DRR output is used, but its [MASK] representation is first enriched with the auxiliary tasks' features through gated fusion.

Under the same experimental conditions as ConnPrompt, TEPrompt performs better and reports state-of-the-art results on PDTB 3.0. The work suggests that auxiliary prompt tasks can strengthen a main prompt task more effectively than ensembling similar templates.

Key points

  • Input design: each task has its own discrete template and answer space. New special tokens [Arg 1] and [Arg 2] mark the arguments and are shared between the DRR and SSC templates.
  • Two-stage gated fusion: one sigmoid gate merges the [CLS] states of the SSC and ACP templates. A second gate injects that combined vector into the DRR [MASK] state before answer-word prediction.
  • Ablations (BERT and RoBERTa backbones):
    • Each auxiliary task alone (DRR+SSC, DRR+ACP) improves over DRR only, and using both works best.
    • Removing the gates lowers performance.
    • SSC or ACP used as the main predictor underperforms DRR.
  • Case study: the plain DRR prompt misclassified an example as Comparison (64.76%, top answer "however"). TEPrompt correctly chose Contingency (83.59%), raising the probability of "so" from 10.60% to 75.43%.
  • Limitation: both auxiliary tasks depend on PDTB-specific properties, namely plain-word top-level sense labels and manually annotated implicit connectives.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-acl.785 · arXiv 2305.10866

@inproceedings{xiang2023teprompt,
  title = {{TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition}},
  author = {Xiang, Wei and Liang, Chao and Wang, Bang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.785},
  eprint = {2305.10866},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.785}
}

References

33 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jifan Chen, Qi Zhang, Pengfei Liu, and Xuanjing Huang. 2016a. Discourse relations detection via a mixed generative-discriminative framework. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 2921–2927, Phoenix, Arizona, USA. find
  2. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016b. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1726–1735, Berlin, Germany. find our card
  3. Technologies, pages 4171–4186, Minneapolis, MN, USA. find
  4. Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Pengjun Xie, Hai-Tao Zheng, Zhiyuan Liu, Juanzi Li, and Hong-Gee Kim. 2021. Prompt-learning for fine-grained entity typing. arXiv preprint, arXiv:2108.10604:1–12. find
  5. Fengyu Guo, Ruifang He, Jianwu Dang, and Jian Wang. 2020. Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, pages 7822–7829, New York, NY, USA. find
  6. Francisco Guzmán, Shafiq Joty, Lluís Màrquez, and Preslav Nakov. 2014. Using discourse structure improves machine translation evaluation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 687–698, Baltimore, MD, USA. find
  7. Yin Jou Huang and Sadao Kurohashi. 2021. Extractive summarization considering discourse and coreference relations based on heterogeneous graph. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL ’21, pages 3046–3052, Stro… find our card
  8. Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, and Guillaume R Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in neural information processing systems, pages 3167–3175, Lake Tahoe, Nevada, United States. find
  9. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344. find our card
  10. Congcong Jiang, Tieyun Qian, Zhuang Chen, Kejian Tang, Shaohui Zhan, and Tao Zhan. 2021a. Generating pseudo connectives with mlms for implicit discourse relation recognition. In The 18th Pacific Rim International Conference on Artificial Intelligence, pages 113–126, Hanoi, Vietnam. find
  11. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167, Helsinki, Finland. find
  12. Feng Jiang, Yaxin Fan, Xiaomin Chu, Peifeng Li, and Qiaoming Zhu. 2021b. Not just classification: Recognizing implicit discourse relation on joint modeling of classification and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2418–2431, Pu… find our card
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Xiao Li, Yu H… find
  14. Maria Liakata, Simon Dobnik, Shyamasree Saha, Colin Batchelor, and Dietrich Rebholz Schuhmann. 2013. A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Pro… find
  15. Chengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang, and Ming Gao. 2021. Transprompt: Towards an automatic transferable prompting framework for few-shot text classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2792–2802, Punta Cana, Dominic… find
  16. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint, arXiv:2107.13586:1–46. find our card
  17. Bonnie Webber, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2019. The penn discourse treebank 3.0 annotation manual. Philadelphia, University of Pennsylvania, 1:1–81. find
  18. Xin Liu, Jiefu Ou, Yangqiu Song, and Xin Jiang. 2020. On the importance of word and sentence representation learning in implicit discourse relation classification. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pages 3830–3836, Virtual. find
  19. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1224–1233, Austin, Texas, USA. find our card
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. arXiv preprint, arXiv:1907.11692:1–13. find
  21. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, pages 1– 18, New Orleans, LA, USA. find
  22. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  23. Huibin Ruan, Yu Hong, Yang Xu, Zhen Huang, Guodong Zhou, and Min Zhang. 2020. Interactively-propagative attention learning for implicit discourse relation recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3168–3178, Online. find our card
  24. Attapol Rutherford, Vera Demberg, and Nianwen Xue. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 281–291, Valencia, Spain. find our card
  25. Ronald Seoh, Ian Birle, Mrinal Tak, Haw-Shiuan Chang, Brian Pinette, and Alfred Hough. 2021. Open aspect target sentiment classification with natural language prompts. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6311–6322, Punta Cana, Dominican Re… find
  26. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… find our card
  27. Wei Xiang and Bang Wang. 2023. A survey of implicit discourse relation recognition. ACM Computing Surveys, 1:1–34. find
  28. Wei Xiang, Bang Wang, Lu Dai, and Yijun Mo. 2022a. Encoding and fusing semantic connection and linguistic evidence for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3247–3257, Dublin, Ireland. find our card
  29. Wei Xiang, Zhenglin Wang, Lu Dai, and Bang Wang. 2022b. ConnPrompt: Connective-cloze prompt learning for implicit discourse relation recognition. In Proceedings of the 29th International Conference on Computational Linguistics, pages 902–911, Gyeongju, Republic of Korea. find our card
  30. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2230–2235, Lisbon, Portugal. find our card
  31. Yingxue Zhang, Fandong Meng, Li Peng, Jian Ping, and Jie Zhou. 2021. Context tracking network: Graph-based context modeling for implicit discourse relation recognition. In Proceedings of the 2021 Conference of the North American Chapter of the Association Computational Linguistics, pages 1592–1599,… find our card
  32. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, page 207–212, Berlin, Germany. find our card
  33. ACL 2023 Responsible NLP Checklist find

↑ back to top