Elamur Research⌕
☰
home / papers / 2019 / Employing the Correspondence of Relations and Connectives to Identify Implicit Discourse Relations via Label Embeddings

Employing the Correspondence of Relations and Connectives to Identify Implicit Discourse Relations via Label Embeddings

Summary

The paper addresses implicit discourse relation recognition (IDRR): deciding how two adjacent text spans relate (e.g., Cause, Contrast) when no explicit connective links them. In the Penn Discourse Treebank (PDTB), annotators insert an "implicit connective" for each argument pair, and each connective maps to a small set of relations. Earlier multi-task models predicted both labels through shared parameters but did not use this mapping.

The authors build on the encoder of Bai and Zhao (2018), which combines word2vec, ELMo and subword inputs with gated, residual CNNs and co-attention. They replace the plain softmax heads with learned embedding matrices for relations and connectives in a shared space. Three penalty terms shape these embeddings:

  • one pulls each connective toward the relations it can express;
  • one keeps connectives of the same relation orthogonal, which prevents collapse;
  • one makes relation embeddings mutually diverse.

The model beats its base system on every PDTB setting and sets new state-of-the-art results for multi-class classification. This shows that label embeddings can carry knowledge from auxiliary labels to the main task.

Key points

  • Data: PDTB, where 53% of implicit connectives correspond to exactly one relation and 44% to two. Connective labels are needed only during training.
  • Results on multi-class settings (vs. Bai and Zhao, 2018):
    • 4-way F1: 53.00 vs 51.06
    • PDTB-Lin 11-way accuracy: 46.48 vs 45.73
    • PDTB-Ji 11-way accuracy: 49.95 vs 48.22
    • All gains are significant at p < 0.05.
  • Binary one-vs-other F1: best reported scores for Comparison (48.44) and Expansion (73.66). Contingency (56.84) and Temporal (38.60) are not the top results.
  • Ablation (4-way F1): adding embeddings alone gives 51.62, a small gain. Every penalty term contributes, and using all three is best.
  • Setup: embedding size d = 80, Adagrad with learning rate 0.001, and loss weights tuned separately for each task setting.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1411

@inproceedings{nguyen2019employing,
  title = {{Employing the Correspondence of Relations and Connectives to Identify Implicit Discourse Relations via Label Embeddings}},
  author = {Nguyen, Linh The and Ngo, Linh Van and Than, Khoat and Nguyen, Thien Huu},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1411},
  url = {https://doi.org/10.18653/v1/p19-1411}
}

References

43 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. In COLING. find our card
  2. Chloé Braud and Pascal Denis. 2015. Comparing word representations for implicit discourse relation classification. In EMNLP. find our card
  3. Deng Cai and Hai Zhao. 2017. Pair-aware neural sentence modeling for implicit discourse relation classification. In IEA/AIE. find
  4. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Implicit discourse relation detection via a deep architecture with gated relevance network. In ACL. find our card
  5. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In NAACL-HLT. find our card
  6. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In EMNLP. find
  7. Wenqiang Lei, Yuanxin Xiang, Yuwei Wang, Qian Zhong, Meichun Liu, and Min-Yen Kan. 2018. Linguistic properties matter for implicit discourse relation recognition: Combining semantic interaction, topic continuity and attribution. In AAAI. find
  8. Maria Liakata, Simon Dobnik, Shyamasree Saha, Colin Batchelor, and Dietrich Rebholz-Schuhmann. 2013. A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task. In EMNLP. find
  9. Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In ICRL. find our card
  10. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In EMNLP. find our card
  11. Yang Liu and Sujian Li. 2016b. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In EMNLP. find our card
  12. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In AAAI. find
  13. Fengyu Guo, Ruifang He, Di Jin, Jianwu Dang, Long-biao Wang, and Xiangang Li. 2018. Implicit discourse relation recognition using neural tensor network with interactive attention and sparse learning. In COLING. find our card
  14. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. In NIPS. find
  15. Peter Jansen, Mihai Surdeanu, and Peter Clark. 2014. Discourse complements lexical semantics for non-factoid answer reranking. In ACL. find
  16. Minh Nguyen and Thien Huu Nguyen. 2018b. Who is killed by police: Introducing supervised attention for hierarchical lstms. In COLING. find
  17. Yangfeng Ji and Jacob Eisenstein. 2015b. One vector is not enough: Entity-augmented distributed semantics for discourse relations. In TACL. find our card
  18. Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grishman. 2016a. Joint event extraction via recurrent neural networks. In NAACL. find
  19. Yangfeng Ji, Gongbo Zhang, and Jacob Eisenstein. 2015a. Closing the gap: Domain adaptation from explicit to implicit discourse relations. In EMNLP. find our card
  20. Yudai Kishimoto, Yugo Murawaki, and Sadao Kurohashi. 2018. A knowledge-augmented neural network model for implicit discourse relation classification. In COLING. find
  21. Alistair Knott. 2014. A data-driven methodology for motivating a set of coherence relations. In Ph.D. Thesis - The University of Edinburgh. find
  22. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In EMNLP. find our card
  23. Man Lan, Yu Xu, and Zhengyu Niu. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In ACL. find our card
  24. Thien Huu Nguyen, Lisheng Fu, Kyunghyun Cho, and Ralph Grishman. 2016c. A two-stage approach for extending event detection to new types via neural networks. In Proceedings of the 1st ACL Workshop on Representation Learning for NLP (RepL4NLP). find
  25. Thien Huu Nguyen and Ralph Grishman. 2015a. Relation extraction: Perspective from convolutional neural networks. In Proceedings of the 1st NAACL Workshop on Vector Space Modeling for NLP (VSM). find
  26. Thien Huu Nguyen and Ralph Grishman. 2015b. Event detection and domain adaptation with convolutional neural networks. In ACL-IJCNLP. find
  27. Thien Huu Nguyen and Ralph Grishman. 2016d. Combining neural networks and log-linear models to improve relation extraction. In Proceedings of IJCAI Workshop on Deep Learning for Artificial Intelligence (DLAI). find
  28. Thien Huu Nguyen, Avirup Sil, Georgiana Dinu, and Radu Florian. 2016b. Toward mention detection robustness with recurrent neural networks. In Proceedings of IJCAI Workshop on Deep Learning for Artificial Intelligence (DLAI). find
  29. Trung Minh Nguyen and Thien Huu Nguyen. 2019. One for all: Neural joint modeling of entities and events. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI). find
  30. Biao Zhang, Deyi Xiong, jinsong su, Qun Liu, Ron-grong Ji, Hong Duan, and Min Zhang. 2016. Variational neural discourse relation recognizer. In EMNLP. find our card
  31. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In COLING. find our card
  32. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL-HLT. find
  33. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In ACL-AFNLP. find our card
  34. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In LREC. find our card
  35. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016. Implicit discourse relation recognition with context-aware character-enhanced embeddings. In COL-ING. find our card
  36. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016b. A stacking gated neural architecture for implicit discourse relation classification. In EMNLP. find our card
  37. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In ACL. find our card
  38. WenTing Wang, Jian Su, and Chew Lim Tan. 2010. Kernel based discourse relation recognition with temporal ordering information. In EMNLP. find our card
  39. Bonnie Webber, Alistair Knott, Matthew Stone, and Aravind Joshi. 1999. Discourse relations: A structural and presuppositional account using lexicalised tag. In ACL. find our card
  40. Changxing Wu, xiaodong shi, Yidong Chen, Yanzhou Huang, and jinsong su. 2016. Bilingually-constrained synthetic data for implicit discourse relation recognition. In EMNLP. find our card
  41. Changxing Wu, Xiaodong Shi, Yidong Chen, Jinsong Su, and Boli Wang. 2017. Improving implicit discourse relation recognition with discourse-specific word embeddings. In ACL. find our card
  42. Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, and Masaaki Nagata. 2014. Dependency-based discourse parser for single-document summarization. In EMNLP. find our card
  43. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In EMNLP. find our card

↑ back to top