Elamur Research⌕
☰
home / papers / 2016 / Shallow Discourse Parsing Using Convolutional Neural Network

Shallow Discourse Parsing Using Convolutional Neural Network

Summary

This system was built by the Shanghai Jiao Tong University team for the CoNLL 2016 Shared Task on shallow discourse parsing in English and Chinese. It targets sense labeling, especially for Non-Explicit relations. These relations lack a connective, which makes them the weakest step of end-to-end parsers. The authors keep the pipeline of Wang and Lan (2015), the top CoNLL-2015 system, which also serves as their baseline, and replace only the sense classifiers. For Explicit relations, a linear-kernel SVM uses just three connective-centered features, one of which comes from the dependency tree. For Non-Explicit relations, a CNN encodes each argument from word and POS embeddings with shared filters. It then joins the two pooled vectors and classifies the pair with a softmax. The same architecture works for both languages. The new classifiers beat the baseline on development data for both relation types. They also lift overall parser F1 slightly, even though connective detection is weaker. This suggests that learned sentence-pair representations help where hand-crafted features run out.

Key points

  • Explicit classifier. The SVM uses three features: the connective string, its POS, and C-HP (the connective combined with the POS of its dependency parent). On English dev, F1 rose from 90.14 (baseline) to 92.11. Chinese dev reached 97.43.
  • CNN design. The model uses wide convolution, one-max pooling, cross-entropy loss with L2 regularization, and AdaGrad. Inputs are 300-d word2vec word vectors plus 50-d POS vectors, with a maximum length of 80. Filters are (4,8,12) with 512 maps for original arguments and (3,3,3) with 1024 maps for refined arguments.
  • Non-Explicit dev results. English F1 improved from 42.92 to 45.50 (+2.58). Chinese dev scored 71.57.
  • Full parser results. F1 gains over the CoNLL-2015 winner were +0.55 on the WSJ test set and +0.61 on the blind test set.
  • Supplementary task (gold arguments), Explicit / Non-Explicit F1:
    • WSJ test: 89.59 / 38.20
    • English blind: 75.74 / 35.38
    • CTB test: 93.19 / 67.41
    • Chinese blind: 74.73 / 56.35

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k16-2010

@inproceedings{qin2016shallow,
  title = {{Shallow Discourse Parsing Using Convolutional Neural Network}},
  author = {Qin, Lianhui and Zhang, Zhisong and Zhao, Hai},
  booktitle = {Proceedings of the CoNLL-16 shared task},
  year = {2016},
  doi = {10.18653/v1/k16-2010},
  url = {https://doi.org/10.18653/v1/k16-2010}
}

References

32 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tsuyoshi Okita, Longyue Wang, and Qun Liu. 2015. The dcu discourse parser: A sense classification task. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 71–77, Beijing, China, July. Association for Computational Linguistics. find
  2. Deng Cai and Hai Zhao. 2016. Neural word segmentation learning for Chinese. In Proceedings of ACL, Berlin, Germany, August. find
  3. Changge Chen, Peilu Wang, and Hai Zhao. 2015. Shallow discourse parsing using constituent parsing tree. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 37–41, Beijing, China, July. Association for Computational Linguistics. find our card
  4. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind K Joshi. 2008. Easily identifiable discourse relations. find our card
  5. Martin Potthast, Tim Gollub, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, and Benno Stein. 2014. Improving the Reproducibility of PAN’s Shared Tasks: Plagiarism Detection, Author Identification, and Author Profiling. In Evangelos Kanoulas, Mihai Lupu, Paul Clough, Mark Sander-son, Mark Hall… find
  6. Christian Chiarcos and Niko Schenk. 2015. A minimalist approach to shallow discourse parsing and implicit relation recognition. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 42–49, Beijing, China, July. Association for Computational Lingu… find
  7. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121–2159. find
  8. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The penn discourse treebank 2.0. In LREC. Citeseer. find our card
  9. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188. find our card
  10. Yangqiu Song, Haoruo Peng, Parisa Kordjamshidi, Mark Sammons, and Dan Roth. 2015. Improving a pipeline architecture for shallow discourse parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 78–83, Beijing, China, July. Association for… find
  11. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. find our card
  12. Majid Laali, Elnaz Davoodi, and Leila Kosseim. 2015. The clac discourse parser at CoNLL-2015. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 56–60, Beijing, China, July. Association for Computational Linguistics. find
  13. Evgeny Stepanov, Giuseppe Riccardi, and Ali Orkan Bayer. 2015. The unitn discourse parser in CoNLL 2015 shared task: Token-level sequence labeling with argument-specific models. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 25–31, Beijing… find our card
  14. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 343–351. Association for Computational Linguistics. find our card
  15. Jia Sun, Peijia Li, Weiqun Xu, and Yonghong Yan. 2015. A shallow discourse parsing system based on maximum entropy model. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 84–88, Beijing, China, July. Association for Computational Linguistics… find
  16. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A pdtb-styled end-to-end discourse parser. Natural Language Engineering, 20(02):151–184. find
  17. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  18. Jianxiang Wang and Man Lan. 2015. A refined end-to-end discourse parser. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 17–24, Beijing, China, July. Association for Computational Linguistics. find our card
  19. Son Nguyen, Quoc Ho, and Minh Nguyen. 2015. Jaist: A two-phase machine learning approach for identifying discourse relations in newswire texts. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 66–70, Beijing, China, July. Association for Com… find
  20. Rui Wang, Masao Utiyama, Isao Goto, Eiichro Sumita, Hai Zhao, and Bao-Liang Lu. 2013. Converting continuous-space language models into n-gram language models for statistical machine translation. find
  21. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 1507–1514. Association for Computational Linguis… find our card
  22. In Proceedings of EMNLP, pages 845–850, Seattle, Washington, USA, October. find
  23. Rui Wang, Hai Zhao, Bao-Liang Lu, Masao Utiyama, and Eiichiro Sumita. 2014. Neural network based bilingual language model growing for statistical machine translation. In Proceedings of EMNLP, pages 189–195, Doha, Qatar, October. find
  24. Peilu Wang, Yao Qian, Frank Soong, Lei He, and Hai Zhao. 2016. Learning distributed word representations for bidirectional lstm recurrent neural network. In Proceedings of NAACL, June. find
  25. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi PrasadO Christopher Bryant, and Attapol T Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of CoNLL, page 2. find our card
  26. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Bonnie Webber, Attapol Rutherford, Chuan Wang, and Hongmin Wang. 2016. The CoNLL-2016 shared task on multilingual shallow discourse parsing. In Proceedings of the Twentieth Conference on Computational Natural Language Learning - Shared Task, Berlin, Germany,… find our card
  27. Yasuhisa Yoshida, Katsuhiko Hayashi, Tsutomu Hirao, and Masaaki Nagata. 2015. Hybrid approach to pdtb-styled discourse parsing for CoNLL-2015. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 95–99, Beijing, China, July. Association for Comp… find
  28. Ye Zhang and Byron Wallace. 2015. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820. find
  29. Zhisong Zhang and Hai Zhao. 2016. Probabilistic graph-based dependency parsing with convolutional neural network. In Proceedings of ACL, Berlin, Germany, August. find
  30. Hai Zhao, Wenliang Chen, and Chunyu Kit. 2009a. Semantic dependency parsing of NomBank and PropBank: An efficient integrated approach via a large-scale feature selection. In Proceedings of EMNLP, pages 30–39, Singapore, August. find our card
  31. Hai Zhao, Wenliang Chen, Chunyu Kity, and Guodong Zhou. 2009b. Multilingual dependency learning: A huge feature engineering method to semantic dependency parsing. In Proceedings of CoNLL, pages 55–60, Boulder, Colorado, June. find
  32. Hai Zhao, Xiaotian Zhang, and Chunyu Kit. 2013. Integrative semantic dependency parsing via efficient large-scale feature selection. Journal of Artificial Intelligence Research, 46:203–233. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qin-2016-shallow-discourse-parsing-convolutional-neural/