Elamur Research⌕
☰
home / papers / 2024 / What Causes the Failure of Explicit to Implicit Discourse Relation Recognition?

What Causes the Failure of Explicit to Implicit Discourse Relation Recognition?

Summary

Classifiers for implicit discourse relations are often trained on explicit examples with their connectives removed, but they do poorly on genuine implicit data. Earlier work blamed linguistic differences between the two types without testing that claim across a corpus. Liu, Wan and Strube identify one concrete cause: label shift. Once a connective is deleted, the remaining arguments may express a different relation than the original annotation. For example, a Temporal example marked by "then" can read as Cause without it.

The authors confirm the shift through a small manual study and then at corpus scale on PDTB 2.0 and PDTB 3.0. Among the factors they examine, the syntactic role of connectives contributes most to the shift. They propose two remedies:

  • a cosine-similarity-based label shift metric, used to filter out noisy training examples;
  • joint learning, in which a RoBERTa-based model recovers a connective and then classifies the relation.

Combining both methods outperforms strong baselines on PDTB and on the RST-annotated GUM corpus. This suggests that cheap explicit data can be used more reliably for implicit relation recognition.

Key points

  • Manual check: Two trained annotators labelled 100 randomly sampled PDTB 2.0 explicit instances with connectives removed. This gave initial evidence of label shift before the corpus-level analysis.
  • Setup: PDTB sections 2–20 were used for training, 0–1 for development and 21–22 for testing. Evaluation covered top- and second-level relations, with RoBERTa base plus a linear layer as the classifier. Code is available at github.com/liuwei1206/Exp2Imp.
  • Ablation on PDTB: Dropping the filtering step lowered top-level F1 by 5.86% on PDTB 2.0 and 4.98% on PDTB 3.0. Removing joint learning also hurt performance, and the two strategies proved complementary.
  • GUM results (v9, DISRPT 2023): Explicit and implicit examples were separated with a heuristic rule. Training on explicit data (E2I-Entire) reached 32.52 F1, compared with 56.81 when training on implicit data. The full method reached 37.56 F1, about 5 points above the E2I baseline.
  • Limitations: Only English PDTB- and RST-style corpora were tested. Other frameworks (e.g., SDRT) and other languages are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.naacl-long.150 · arXiv 2404.00999

@inproceedings{liu2024what,
  title = {{What Causes the Failure of Explicit to Implicit Discourse Relation Recognition?}},
  author = {Liu, Wei and Wan, Stephen and Strube, Michael},
  booktitle = {Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.150},
  eprint = {2404.00999},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2024.naacl-long.150}
}

References

52 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nicholas Asher, Nicholas Michael Asher, and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. find
  2. Sasha Blair-Goldensohn, Kathleen McKeown, and Owen Rambow. 2007. Building and refining rhetorical-semantic relation models. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 428–435, Rochester, New York. Association for Computational Linguistics. ↗
  3. Hardarik Blühdorn. 2008. Subordination and coordination in syntax semantics and discourse: Evidence from the study of connectives. ↗
  4. Chloé Braud, Yang Janet Liu, Eleni Metheniti, Philippe Muller, Laura Rivière, Attapol Rutherford, and Amir Zeldes. 2023. find
  5. The DISRPT 2023 shared task on elementary discourse unit segmentation, connective detection, and relation classification. In Proceedings of the 3rd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2023), pages 1–21, Toronto, Canada. The Association for Computational Linguistics. find our card
  6. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurovsky. 2001. Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. ↗ our card
  7. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics. find our card
  8. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 785–794, New York, NY, USA. Association for Computing Machinery. ↗
  9. Matt Ellis. 2023. How to use conjunctive adverbs. Grammarly.com. find
  10. David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2003. English gigaword. Linguistic Data Consortium, Philadelphia, 4(1):34. find
  11. Hsin-Ping Huang and Junyi Jessy Li. 2019. Unsupervised adversarial domain adaptation for implicit discourse relation classification. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 686–695, Hong Kong, China. Association for Computational Linguistics. find
  12. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344. ↗ our card
  13. Yangfeng Ji, Gongbo Zhang, and Jacob Eisenstein. 2015. Closing the gap: Domain adaptation from explicit to implicit discourse relations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2219–2224, Lisbon, Portugal. Association for Computational Linguistics. find our card
  14. Congcong Jiang, Tieyun Qian, Zhuang Chen, Kejian Tang, Shaohui Zhan, and Tao Zhan. 2021. Generating pseudo connectives with mlms for implicit discourse relation recognition. In PRICAI 2021: Trends in Artificial Intelligence, pages 113–126, Cham. Springer International Publishing. find
  15. Najoung Kim, Song Feng, Chulaka Gunasekara, and Luis Lastras. 2020. Implicit discourse relation classification: We need to talk about evaluation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5404– 5414, Online. Association for Computational Linguistics. ↗ our card
  16. Yudai Kishimoto, Yugo Murawaki, and Sadao Kurohashi. 2020. Adapting BERT to implicit discourse relation classification with a focus on discourse connectives. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1152– 1158, Marseille, France. European Language Resources Association. find our card
  17. Murathan Kurfalı and Robert Östling. 2019. Zero-shot transfer for implicit discourse relation classification. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 226–231, Stockholm, Sweden. Association for Computational Linguistics. find our card
  18. Murathan Kurfalı and Robert Östling. 2021. Let’s be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction. In Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language, pages 1–10, Online. Association for Computational Linguistics. ↗ our card
  19. Mirella Lapata and Alex Lascarides. 2004. Inferring sentence-internal temporal relations. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 153–160, Boston, Massachusetts, USA. Association for Computational Linguistics. find our card
  20. Wei Liu and Michael Strube. 2023. Annotation-inspired implicit discourse relation classification with auxiliary discourse connective generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15696–15712, Toronto, Canada. Association for Computational Linguistics. ↗ our card
  21. Yang Janet Liu, Tatsuya Aoyama, and Amir Zeldes. 2023c. What’s hard in English RST parsing? predictive models for error analysis. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 31–42, Prague, Czechia. Association for Computational Linguistics. find
  22. Wanqiu Long and Bonnie Webber. 2022. Facilitating contrastive learning of discourse relational senses by exploiting the hierarchy of sense relations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10704– 10716, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. ↗ our card
  23. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text - Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. ↗ our card
  24. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 368–375, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
  25. Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014. Text-level discourse dependency parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25–35, Baltimore, Maryland. Association for Computational Linguistics. ↗
  26. Eleni Metheniti, Chloé Braud, Philippe Muller, and Laura Rivière. 2023. DisCut and DiscReT: MELODI at DISRPT 2023. In Proceedings of the 3rd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2023), pages 29–42, Toronto, Canada. The Association for Computational Linguistics. find
  27. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 343–351, Singapore. Association for Computational Linguistics. find our card
  28. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 13–16, Suntec, Singapore. Association for Computational Linguistics. find our card
  29. Wei Liu, Yi Fan, and Michael Strube. 2023a. HITS at DISRPT 2023: Discourse segmentation, connective detection, and relation classification. In Proceedings of the 3rd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2023), pages 43–49, Toronto, Canada. The Association for Computational Linguistics. find
  30. Wei Liu, Xiyan Fu, and Michael Strube. 2023b. Modeling structural similarities between documents for coherence assessment with graph convolutional networks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7792–7808, Toronto, Canada. Association for Computational Linguistics. find
  31. R. Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind K. Joshi, Livio Robaldo, and Bonnie Lynn Webber. 2006a. The penn discourse treebank 2.0 annotation manual. find our card
  32. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA). ↗ our card
  33. Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, and Bonnie Webber. 2006b. The penn discourse treebank 1.0 annotation manual. find our card
  34. Rashmi Prasad, Bonnie Webber, and Alan Lee. 2018. Discourse annotation in the PDTB: The next generation. In Proceedings 14th Joint ACL - ISO Workshop on Interoperable Semantic Annotation, pages 87–97, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  35. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1006–1017, Vancouver, Canada. Association for Computational Linguistics. find our card
  36. Manami Saito, Kazuhide Yamamoto, and Satoshi Sekine. 2006. Using phrasal patterns to identify discourse relations. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 133–136, New York City, USA. Association for Computational Linguistics. find our card
  37. Erbo Shang, Xiaohua Liu, Hailong Wang, Yangfeng Rong, and Yuerong Liu. 2019. Research on the application of artificial intelligence and distributed parallel computing in archives classification. In 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pages 1267–1271. find
  38. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations: an assessment. Natural Language Engineering, 14(3):369–416. find
  39. C.E. Sporleder and A. Lascarides. 2005. Exploiting linguistic cues to classify rhetorical relations. In Proceedings of Recent Advances in Natural Language Processing (RANLP-05), pages 532–539. Unknown Publisher. Pagination: 8. find
  40. Catherine Traffis. 2021. What are conjunctions? definition and examples. Grammarly.com. Computational Semantics - Long Papers, pages 134– 141, Gothenburg, Sweden. Association for Computational Linguistics. find
  41. Bonnie Webber, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2019b. The penn discourse treebank 3.0 annotation manual. Philadelphia, University of Pennsylvania, 35:108. find our card
  42. Wei Xiang, Chao Liang, and Bang Wang. 2023. TEPrompt: Task enlightenment prompt learning for implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12403–12414, Toronto, Canada. Association for Computational Linguistics. find our card
  43. Amir Zeldes. 2017. The GUM corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3):581–612. ↗
  44. Amir Zeldes, Yang Janet Liu, Mikel Iruskieta, Philippe Muller, Chloé Braud, and Sonia Badene. 2021. find
  45. The DISRPT 2021 shared task on elementary discourse unit segmentation, connective detection, and relation classification. In Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2021), pages 1–12, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  46. Hao Zhou, Man Lan, Yuanbin Wu, Yuefeng Chen, and Meirong Ma. 2022. Prompt-based connective prediction method for fine-grained implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3848–3858, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find our card
  47. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Coling 2010: Posters, pages 1507–1514, Beijing, China. Coling 2010 Organizing Committee. find our card
  48. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579–2605. find
  49. Hanna Varachkina and Franziska Pannach. 2021. A unified approach to discourse relation classification in nine languages. In Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2021), pages 46–50, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  50. Xun Wang, Sujian Li, Jiwei Li, and Wenjie Li. 2012. Implicit discourse relation recognition by selecting typical training examples. In Proceedings of COL-ING 2012, pages 2757–2772, Mumbai, India. find our card
  51. The COLING 2012 Organizing Committee. find
  52. Bonnie Webber, Rashmi Prasad, and Alan Lee. 2019a. Ambiguity in explicit discourse connectives. In Proceedings of the 13th International Conference on PDTB 2.0 Comparison Contingency Expansion Temporal Comparison.Concession Comparison.Contrast Contingency.Cause Contingency.Pragmatic cause Expansion.Conjunction Expansion.Instantiation Expansion.Alternative Expansion.List Expansion.Restatement Temporal.Asynchronous Temporal.Synchrony Dataset PDTB 3.0 Comparison Contingency Expansion Temporal Comparison.Concession Comparison.Contrast Contingency.Cause Contingency.Cause+Belief Contingency.Condition Contingency.Purpose Expansion.Conjunction Expansion.Equivalence Expansion.Instantiation Expansion.Level-of-detail Expansion.Manner Expansion.Substitution Temporal.Asynchronous Temporal.Synchronous PDTB 2.0 PDTB 3.0 GUM Adversative Explanation Context Contingency Dev 1462 1183 1944 1653 276 1607 Test 1285 1046 1767 1474 264 1619 Causal *Table 5: Relations of the GUM dataset used in this work.* find our card

↑ back to top

This page in Markdown →