Elamur Research⌕
☰
home / papers / 2019 / Discourse Relation Prediction: Revisiting Word Pairs with Convolutional Networks

Discourse Relation Prediction: Revisiting Word Pairs with Convolutional Networks

Summary

This work addresses discourse relation classification in the Penn Discourse Tree Bank (PDTB). The focus is implicit relations, where no connective such as "because" signals the link between two text spans. Earlier systems used cross-argument word pairs as discrete features, which produced very sparse feature spaces. The authors instead feed a CNN the Cartesian product of words from Arg1 and Arg2, in both directions, with word and POS embeddings concatenated. Even-length filters with a stride of two capture single word pairs or sequences of k pairs (WP-k), so one word can be aligned with a short phrase in the other argument. A second CNN extracts n-gram features from each argument separately, and gating layers merge the two components. Implicit and explicit relations are learned jointly, each with its own output layer. The model sets a new state of the art on four-way classification for both relation types and on one binary task. It also needs roughly half the parameters and training time of a strong two-layer Bi-LSTM-CRF. This shows that a compact, parallelizable, and partly interpretable model can compete with heavier recurrent architectures.

Key points

  • On four-way explicit relation classification, the model cuts the error by about 20%.
  • Compared with a two-layered Bi-LSTM-CRF baseline, it has about half the learnable parameters and needs roughly half the training time.
  • The word-pair representation drops function words shorter than 3 characters. Convolution weights are shared between the Arg1×Arg2 and Arg2×Arg1 inputs. The model uses 100 feature maps per word-pair filter and 50 per n-gram filter.
  • In an ablation, removing the gates reduced the mean F1 over three runs from 50.9 to 50.1. Joint training with explicit relations also improved implicit relation prediction.
  • Analysis of the learned features showed two patterns. Small filters tend to pick verb–verb or adjective–adjective pairs, such as (bumped : slashed). Larger filters align a key word with a phrase in the other argument, such as (maximum : spending limits).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-5951

@inproceedings{varia2019discourse,
  title = {{Discourse Relation Prediction: Revisiting Word Pairs with Convolutional Networks}},
  author = {Varia, Siddharth and Hidey, Christopher and Chakrabarty, Tuhin},
  booktitle = {Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2019},
  doi = {10.18653/v1/w19-5951},
  url = {https://doi.org/10.18653/v1/w19-5951}
}

References

25 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. tionship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1299– 1308. Association for Computational Linguistics. find
  2. Fatemeh Torabi Asr and Vera Demberg. 2015. Uniform information density at the level of discourse relations: Negation markers and discourse connective omission. In IWCS 2015, page 118. find
  3. Wenqiang Lei, Xuancong Wang, Meichun Liu, Ilija Ilievski, Xiangnan He, and Min-Yen Kan. 2017. Swim: A simple word interaction model for implicit discourse relation recognition. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 4026–4032. find
  4. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. CoRR, abs/1807.05154. find our card
  5. Or Biran and Kathleen McKeown. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 69–73, Sofia, Bulgaria. Association for Computational Linguist… find our card
  6. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. CoRR, abs/1609.06380. find our card
  7. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 2750–2756. AAAI Press. find
  8. Sasha Blair-Goldensohn, Kathleen McKeown, and Owen Rambow. 2007. Building and refining rhetorical-semantic relation models. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 42… find
  9. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 368–375, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
  10. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1726– 1735, B… find our card
  11. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 683–691, S… find our card
  12. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find our card
  13. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind Joshi. 2008. Easily identifiable discourse relations. In Coling 2008: Companion volume: Posters, pages 87–90, Manchester, UK. find our card
  14. Coling 2008 Organizing Committee. find
  15. Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In Arxiv. Google Research. find
  16. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In In Proceedings of LREC. find our card
  17. Yangfeng Ji, Gongbo Zhang, and Jacob Eisenstein. 2015. Closing the gap: Domain adaptation from explicit to implicit discourse relations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2219– 2224, Lisbon, Portugal. Association for Computational Lingui… find our card
  18. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2263–2270. Association for Computational Linguistics. find our card
  19. Diederik P. Kingma and Jimmy Ba. 2014. A method for stochastic optimization. abs/1412.6980. find
  20. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1006– 1017, Vanc… find our card
  21. Adam: CoRR, Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse rela-Hannah Rohde and William S. Horton. 2010. Why or what next? eye movements reveal expectations about discourse direction. find
  22. Samuel Rönnqvist, Niko Schenk, and Christian Chiarcos. 2017. A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 256–262, Vanco… find our card
  23. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1):1929– 1958. find
  24. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Attapol Rutherford, Bonnie Webber, Chuan Wang, and Hongmin Wang. 2016. find
  25. CoNLL 2016 shared task on multilingual shallow discourse parsing. In Proceedings of the CoNLL-16 shared task, pages 1– 19, Berlin, Germany. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/varia-2019-discourse-relation-prediction-revisiting-word/