Elamur Research⌕
☰
home / papers / 2018 / Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph

Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph

Summary

Implicit discourse relations, which have no connective such as "because", are hard to classify. Most earlier systems examine only the two discourse units (DUs) involved and ignore the rest of the paragraph. Dai and Huang propose a paragraph-level neural model for the PDTB corpus. It encodes a whole paragraph at once and labels every relation between adjacent DUs, both implicit and explicit.

The architecture has two levels:

  • A word-level Bi-LSTM, followed by max-pooling, produces one vector per DU.
  • A second, DU-level Bi-LSTM lets each unit's representation absorb information from its neighbours.

Two additions improve results further. Separate (untied) prediction layers handle explicit and implicit relations, and a CRF layer captures typical sequences of relation labels. The full model reaches 48.82% macro-F1 on four-way implicit classification, beating the strongest prior system. Explicit relations also improve slightly.

The work shows that wider context and the paragraph's overall discourse structure help recognize difficult implicit relations. Gains appear even for short paragraphs.

Key points

  • Input embeddings are 343-dimensional: 300-d Google word2vec, 36-d one-hot POS tags and 7-d named entity tags, all produced with Stanford CoreNLP.
  • On implicit relations, the full model's 48.82% macro-F1 beats Lan et al. (2017) by about 1 point and neural tensor models such as Lei et al. (2017) by over 2 points.
  • In binary classification, the model with untied parameters gains about 6 F1 points on Comparison and 7 on Temporal over the best previous systems.
  • Majority-vote ensembles of 10 runs raise macro-F1 to 51.84% for implicit and 94.17% for explicit relations.
  • Paragraph-level variants beat DU-pair baselines across all paragraph lengths. Gains are largest for paragraphs with more than 5 DUs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

31 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1013 · arXiv 1804.05918

@inproceedings{dai2018improving,
  title = {{Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph}},
  author = {Dai, Zeyu and Huang, Ruihong},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1013},
  eprint = {1804.05918},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-1013}
}

References

39 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 1310– 1319. find
  2. Fatemeh Torabi Asr and Vera Demberg. 2012. Implicitness of discourse relations. In Coling. pages 2669– 2684. find our card
  3. Alex Lascarides and Nicholas Asher. 1993. Temporal interpretation, discourse relations and commonsense entailment. Linguistics and philosophy 16(5):437– 493. find
  4. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, Springer, pages 85–112. find our card
  5. Wenqiang Lei, Xuancong Wang, Meichun Liu, Ilija Ilievski, Xiangnan He, and Min-Yen Kan. 2017. Swim: A simple word interaction model for implicit discourse relation recognition. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. pages 4026–4032. ht… find
  6. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Implicit discourse relation detection via a deep architecture with gated relevance network. In ACL 2016. find our card
  7. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, De… find
  8. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. pages 362–371. find our card
  9. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1. Association for Computational Linguistics, Stroudsburg, PA, USA, EM… find our card
  10. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing coadaptation of feature detectors. arXiv preprint arXiv:1207.0580 . find
  11. Jerry R Hobbs. 1985. On the coherence and structure of discourse. CSLI. find
  12. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A pdtb-styled end-to-end discourse parser. Natural Language Engineering 20(2):151–184. find
  13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. find
  14. Yang Liu and Mirella Lapata. 2017. Learning contextually informed representations for linear-time discourse parsing. In EMNLP. find our card
  15. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. TACL 3:329–344. https://tacl2013.cs.columbia.edu/ojs/ index.php/tacl/article/view/536. ↗ our card
  16. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. pages 1224– 1233. ht… find our card
  17. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. In Proceedings of NAACL-HLT. pages 332–342. find
  18. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 . find
  19. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. volume 951, pages 282–289. find
  20. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA.. pages 2750–2756. http://www.aaai.org/ocs/in… ↗
  21. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Pro-Ramesh Nallapati, Bowen Zhou, Cicero Nogueira dos santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstrac… find our card
  22. Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI. pages 3776–3784. find our card
  23. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 -Volume… find our card
  24. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. pages 3104–3112. find
  25. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi PrasadO Christopher Bryant, and Attapol T Rutherford. 2015. The conll-2015 shared task on shallow discourse parsing. find our card
  26. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers. Association for Computational Linguistics, pages 13–16. find our card
  27. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In EMNLP. The Association for Computational Linguistics, pages 2230–2235. find our card
  28. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind Joshi. 2008. Easily identifiable discourse relations. In In Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) Short Papers. find our card
  29. Ruqing Zhang, Jiafeng Guo, Yanyan Lan, Jun Xu, and Xueqi Cheng. 2017a. Spherical paragraph model. arXiv preprint arXiv:1707.05635 . — [arXiv:1707.05635](https://arxiv.org/abs/1707.05635) find
  30. R. Prasad, N. Dinesh, Lee A., E. Miltsakaki, L. Robaldo, Joshi A., and B. Webber. 2008a. The Penn Discourse Treebank 2.0. In lrec2008. find our card
  31. Yizhe Zhang, Dinghan Shen, Guoyin Wang, Zhe Gan, Ricardo Henao, and Lawrence Carin. 2017b. De-convolutional paragraph representation learning. In Advances in Neural Information Processing Systems. pages 4170–4180. find
  32. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008b. The penn discourse treebank 2.0. In In Proceedings of LREC. find our card
  33. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). volume 2… find our card
  34. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016. A stacking gated neural architecture for implicit discourse relation classification. In EMNLP. pages 2263–2270. find our card
  35. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric P. Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - Augu… find our card
  36. Attapol Rutherford and Nianwen Xue. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In HLT-NAACL. pages 799–808. find our card
  37. Attapol T Rutherford, Vera Demberg, and Nianwen Xue. 2016. Neural network models for implicit discourse relation classification in english and chinese without surface features. arXiv preprint arXiv:1606.01990 . — [arXiv:1606.01990](https://arxiv.org/abs/1606.01990) find
  38. Attapol T Rutherford and Nianwen Xue. 2016. Robust non-explicit neural discourse parser in english and chinese. find our card
  39. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45(11):2673–2681. find

↑ back to top

This page in Markdown →