Elamur Research⌕
☰
home / papers / 2016 / Implicit Discourse Relation Classification via Multi-Task Neural Networks

Implicit Discourse Relation Classification via Multi-Task Neural Networks

Summary

When no explicit connective such as "but" or "because" is present, identifying the discourse relation between two text spans is hard. Such implicit relations are common in real text and remain a major weakness of discourse parsers. The authors note that existing annotated corpora are small and are usually used in isolation, which makes data sparsity worse. They argue that corpora built under different frameworks still share related relation senses. They therefore combine three sources in a multi-task neural network (MTNN): PDTB, RST-DT, and an automatically built corpus from the New York Times. Implicit PDTB classification is the main task. Explicit PDTB relations, RST-DT relations and connective-word prediction on NYT serve as auxiliary tasks. Argument pairs are encoded by a CNN that convolves over pairs of word windows, one taken from each argument, and then applies dynamic pooling. Each task learns both its own representations and shared ones. Multi-task training clearly beats a single-task version and reaches state-of-the-art results on three of the four top-level PDTB classes. This shows that heterogeneous discourse resources can complement one another.

Key points

  • Data: PDTB has 2159 WSJ articles and is evaluated on 4 top-level classes. From RST-DT (385 articles), the authors kept the 12 most frequent of 18 classes, giving 19,681 relations. From the NYT Corpus they built a noisy set of about 40,000 pairs by deleting 35 common connectives.
  • Model: A CNN applies tanh filters to concatenated window pairs, one window from each argument. Dynamic pooling reduces the resulting feature map to a fixed n_p × n_p grid, which keeps positional information. The network is embedded in an MTNN with shared and task-specific layers.
  • Four-way classification: MTL improves over the single-task baseline in Accuracy (52.82 → 57.27) and in F1 (37.65 → 44.98). Both gains are significant (p < 0.05).
  • Binary one-vs-other F1:
    • Temporal reaches 37.17, up from the 33.30 of the best baseline (R&X 2015).
    • Contingency reaches 55.88 and Expansion 69.97.
    • Comparison reaches 37.91 and stays below R&X's 41.00, possibly because this relation relies more on syntax.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics Discourse analysis

 Institutions

🎓Peking UniversityBeijing · China74 works · 31,748 citations

Cite

Cite as Yang Liu, Sujian Li, Xiaodong Zhang and Zhifang Sui. 2016. Implicit Discourse Relation Classification via Multi-Task Neural Networks. arXiv:1603.02776.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1603.02776 · arXiv 1603.02776

@misc{liu2016implicit,
  title = {{Implicit Discourse Relation Classification via Multi-Task Neural Networks}},
  author = {Yang Liu, Sujian Li, Xiaodong Zhang and Zhifang Sui},
  year = {2016},
  doi = {10.48550/arxiv.1603.02776},
  eprint = {1603.02776},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1603.02776}
}

References

22 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [2015] Cao, Z.; Wei, F.; Dong, L.; Li, S.; and Zhou, M. 2015. Ranking with recursive neural networks and its application to multi-document summarization. In Proceedings of AAAI. find
  2. [2011] Collobert, R.; Weston, J.; Bottou, L.; Karlen, M.; Kavukcuoglu, K.; and Kuksa, P. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research 12:2493–2537. our card
  3. [2010] Hernault, H.; Bollegala, D.; and Ishizuka, M. 2010. A Semi-Supervised Approach to Improve Classification of Infrequent Discourse Relations Using Feature Vector Extension. In Proceedings of EMNLP, 399–409. our card
  4. [2015] Ji, Y., and Eisenstein, J. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics 3:329–344. our card
  5. [2014] Kim, Y. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP, 1746– 1751. our card
  6. [2003] Klein, D., and Manning, C. D. 2003. Accurate unlexicalized parsing. In Proceedings of ACL, 423–430. our card
  7. [2013] Lan, M.; Xu, Y.; and Niu, Z.-Y. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In Proceedings of ACL, 476– 485. our card
  8. [2014] Li, S.; Wang, L.; Cao, Z.; and Li, W. 2014. Text-level discourse dependency parsing. In Proceedings of ACL, volume 1, 25–35. find
  9. [2009] Lin, Z.; Kan, M.-Y.; and Ng, H. T. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of EMNLP, 343–351. our card
  10. [2015] Liu, X.; Gao, J.; He, X.; Deng, L.; Duh, K.; and Wang, Y.-Y. 2015. Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In Proceedings NAACL, 912–921. our card
  11. [2010] Louis, A.; Joshi, A.; Prasad, R.; and Nenkova, A. 2010. Using entity features to classify implicit discourse relations. In Proceedings of SigDial, 59–62. our card
  12. [1988] Mann, W. C., and Thompson, S. A. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse 8(3):243–281. our card
  13. [2013] McKeown, K., and Biran, O. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of ACL, 69–73. our card
  14. [2012] Park, J., and Cardie, C. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of SigDial, 108–112. our card
  15. [2014] Pennington, J.; Socher, R.; and Manning, C. D. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, 1532–1543. our card
  16. [2009] Pitler, E.; Louis, A.; and Nenkova, A. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of ACL, 683–691. our card
  17. [2007] Prasad, R.; Miltsakaki, E.; Dinesh, N.; Lee, A.; Joshi, A.; Robaldo, L.; and Webber, B. L. 2007. The penn discourse treebank 2.0 annotation manual. find
  18. [2014] Rutherford, A., and Xue, N. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of EACL, 645–654. our card
  19. [2015] Rutherford, A., and Xue, N. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In Proceedings of NAACL, 799–808. our card
  20. [2008] Sandhaus, E. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia 6(12):e26752. find
  21. [2008] Sporleder, C., and Lascarides, A. 2008. Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering 14(03):369– 416. find
  22. [2010] Zhou, Z.-M.; Xu, Y.; Niu, Z.-Y.; Lan, M.; Su, J.; and Tan, C. L. 2010. Predicting discourse connectives for implicit discourse relation recognition. In International Conference on Computational Linguistics, 1507–1514. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2016-implicit-discourse-relation-classification-multi/