Elamur Research⌕
☰
home / papers / 2021 / Not Just Classification: Recognizing Implicit Discourse Relation on Joint Modeling of Classification and Generation

Not Just Classification: Recognizing Implicit Discourse Relation on Joint Modeling of Classification and Generation

Summary

Implicit discourse relation recognition (IDRR) is usually treated as plain label classification, so models never see what a relation actually means. The authors also cast IDRR as text generation. Their joint model, CG-T5, built on T5, predicts a relation label from encoder states via a span representation layer while the decoder writes a natural-language target sentence expressing the relation. They test three target forms: the bare relation name, an explanation of the relation (placed before or after the name), and a question about one unit, an idea borrowed from QA-style annotation. Long discourse units, such as whole paragraphs, would not fit into such templates. A semantic role labeling–based step therefore compresses each unit into short core information. On the Chinese MCDTB and the English PDTB, CG-T5 beats several state-of-the-art baselines, and the question form works best. The classifier is stronger on frequent classes, while the generator handles rare ones better and gives readable output. Combining both improves accuracy and interpretability.

Key points

  • Corpora: MCDTB has 720 Xinhua News articles with paragraph-level relations (3 classes, 15 types), all treated as implicit. PDTB is evaluated with top-level 4-way classification.
  • PDTB per-class results: The question-form generator reaches 55.40 F1 on Comparison, 8.21 points above BERT. The authors attribute this to cue words like "negate"/"opposite" in the question templates.
  • MCDTB fold 1, classifier output: The joint classifier scores 62.75 micro-F1 at second level and 71.79 at top level.
  • MCDTB fold 1, agreement and oracle: The classifier and generator agree on 91.34% (second level) and 93.51% (top level) of cases. An oracle choosing the correct one would reach 65.00 and 74.19.
  • Generation quality: Rouge-1/2/3/L scores are 83.99/79.69/77.22/82.97, above both vanilla T5 and GPT-2. In ablations, joint training improved both the classification and generation heads.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.emnlp-main.187

@inproceedings{jiang2021not,
  title = {{Not Just Classification: Recognizing Implicit Discourse Relation on Joint Modeling of Classification and Generation}},
  author = {Jiang, Feng and Fan, Yaxin and Chu, Xiaomin and Li, Peifeng and Zhu, Qiaoming},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.187},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.187}
}

References

34 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. In Proceedings of the 27th International Conference on Computational Linguistics (COLING), pages 571–583. find our card
  2. Hongxiao Bai, Hai Zhao, and Junhan Zhao. 2019. Memorizing all for implicit discourse relation recognition. find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  4. Fengyu Guo, Ruifang He, Jianwu Dang, and Jian Wang. 2020. Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition. In The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), pages 7822–7829. find
  5. Fengyu Guo, Ruifang He, Di Jin, Jianwu Dang, Long-biao Wang, and Xiangang Li. 2018. Implicit discourse relation recognition using neural tensor network with interactive attention and sparse learning. In Proceedings of the 27th International Conference on Computational Linguistics (COLING), pages 547… find our card
  6. Ruifang He, Jian Wang, Fengyu Guo, and Yugui Han. 2020. TransS-driven joint learning architecture for implicit discourse relation recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 139– 148. find our card
  7. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics (TACL), 3:329– 344. find our card
  8. Feng Jiang, Peifeng Li, and Qiaoming Zhu. 2019. Joint modeling of recognizing macro Chinese discourse nuclearity and relation based on structure and topic gated semantic network. In Natural Language Processing and Chinese Computing (NLPCC), pages 276–286. find
  9. Feng Jiang, Sheng Xu, Xiaomin Chu, Peifeng Li, Qiaoming Zhu, and Guodong Zhou. 2018. MCDTB: A macro-level Chinese discourse TreeBank. In Proceedings of the 27th International Conference on Computational Linguistics (COLING), pages 3493– 3504. find our card
  10. Najoung Kim, Song Feng, Chulaka Gunasekara, and Luis Lastras. 2020. Implicit discourse relation classification: We need to talk about evaluation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 5404–5414. Association for Computational Linguisti… find our card
  11. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1299–1… find our card
  12. Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020. A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 5849–5859. Association for Computational Linguistics. find our card
  13. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1340–1350. find
  14. Yancui Li, Wenhe Feng, Jing Sun, Fang Kong, and Guodong Zhou. 2014. Building Chinese discourse corpus with connective-driven dependency tree structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2105–2114. Association for Computational Li… find
  15. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1224– 1233. find our card
  16. Linh The Nguyen, Linh Van Ngo, Khoat Than, and Thien Huu Nguyen. 2019. Employing the correspondence of relations and connectives to identify implicit discourse relations via label embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4201… find our card
  17. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC), pages 2961–2968. European Language Resources Associatio… find our card
  18. Valentina Pyatkin, Ayal Klein, Reut Tsarfaty, and Ido Dagan. 2020. QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2804–2819. find our card
  19. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2263–2270. The Association for Computational Linguistics. find our card
  20. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  21. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  22. Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020. Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics (TACL), 8:264– 280. find our card
  23. Huibin Ruan, Yu Hong, Yang Xu, Zhen Huang, Guodong Zhou, and Min Zhang. 2020. Interactively-propagative attention learning for implicit discourse relation recognition. In Proceedings of the 28th International Conference on Computational Linguistics (COLING). International Committee on Computational… find our card
  24. Wei Shi and Vera Demberg. 2019. Learning to explic-itate connectives with seq2seq network for implicit discourse relation classification. In Proceedings of the 13th International Conference on Computational Semantics (IWCS), pages 188–199. find
  25. Zhenhua Sun, Feng Jiang, Peifeng Li, and Qiaoming Zhu. 2020. Macro discourse relation recognition via discourse argument pair graph. In Natural Language Processing and Chinese Computing (NLPCC), pages 108–119. find
  26. Shubham Toshniwal, Haoyue Shi, Bowen Shi, Lingyu Gao, Karen Livescu, and Kevin Gimpel. 2020. A cross-task analysis of text span representations. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 166–176, Online. Association for Computational Linguistics. find
  27. Bonnie Webber, Andrei Popescu-Belis, and Jörg Tiedemann, editors. 2017. Proceedings of the Third Workshop on Discourse in Machine Translation. find
  28. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find
  29. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45. find our card
  30. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6953–6963. Association for Computational Linguistics. find our card
  31. Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-aware neural extractive text summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 5021–5031. Association for Computational Linguistics. find our card
  32. Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, and Guodong Zhou. 2019. Topic tensor network for implicit discourse relation recognition in Chinese. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 608–618. Association for Computational Linguisti… find our card
  33. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2230–2235. Association for Compu… find our card
  34. Yi Zhou, Xiaomin Chu, Qiaoming Zhu, Feng Jiang, and Peifeng Li. 2019. Macro discourse relation classification based on macro semantics representation. Journal of Chinese Information Processing, 33(3):1– 7. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jiang-2021-not-just-classification-recognizing-implicit/