Robust Non-Explicit Neural Discourse Parser in English and Chinese

Summary

Sense classification of implicit and other non-explicit discourse relations is considered the hardest part of shallow discourse parsing. Earlier neural approaches often depended on syntactic parses, large semantic lexicons, or architectures tuned to one label set. For the CoNLL 2016 Shared Task supplementary evaluation, the authors built a deliberately simple classifier for English and Chinese that needs only pre-trained word vectors.

Each argument (Arg1, Arg2) is turned into a single vector by summing its word embeddings. The two vectors are then fed jointly into a feedforward network with two tanh hidden layers and a softmax output. It is trained with cross-entropy and Adagrad in Theano and converges within minutes on one CPU core.

The system placed first on the English blind test set and third on Chinese. It also scored higher on test data than on development data, which suggests robustness to domain shift.

An error analysis indicates that the neural model and surface-feature MaxEnt systems make partly different mistakes. This makes the approach a lightweight option for languages that lack rich linguistic resources.

Key points

Architecture and inputs: two hidden layers of 300 units, about 270k parameters. English uses 300-dimensional Skip-gram Google News vectors. Chinese uses self-trained 250-dimensional vectors from the Gigaword corpus, which beat the 300-dimensional ones.
Blind-test accuracy: 0.3767 for English (ranked first) and 0.6338 for Chinese (ranked third).
Baselines: for English, last year's winning system (Wang and Lan, 2015), a MaxEnt model with millions of features. For Chinese, a Brown cluster pair model with 10,000 selected features.
Chinese (CDTB) test set: the model beats the most-frequent-tag baseline by about 7 points and the Brown cluster baseline by about 3 points (absolute). However, it learns only the three most frequent senses.
Error analysis: confusion pairs were compared using Bayes Factors. The neural model handles Expansion.Instantiation better, while the feature-based system is stronger on Expansion.Conjunction. Code and trained models are available at github.com/attapol/nn_discourse_parser.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k16-2007

@inproceedings{rutherford2016robust,
  title = {{Robust Non-Explicit Neural Discourse Parser in English and Chinese}},
  author = {Rutherford, Attapol and Xue, Nianwen},
  booktitle = {Proceedings of the CoNLL-16 shared task},
  year = {2016},
  doi = {10.18653/v1/k16-2007},
  url = {https://doi.org/10.18653/v1/k16-2007}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Berg-eron, Nicolas Bouchard, and Yoshua Bengio. 2012. Theano: new features and speed improvements. find
Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop. find
Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. In Neural Networks: Tricks of the Trade, pages 437– 478. Springer. find
James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. 2010. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June. Oral… find
David Graff and Ke Chen. 2005. Chinese gigaword. LDC Catalog No.: LDC2003T09, ISBN, 1:58563– 58230. find
Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344. find our card
Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. find
Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume… find our card
Attapol T. Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2014), Gothenburg, Sweden, Ap… find our card
A. T. Rutherford, V. Demberg, and N. Xue. 2016. Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features. ArXiv e-prints, June. find
Jianxiang Wang and Man Lan. 2015. A refined end-to-end discourse parser. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 17–24, Beijing, China, July. Association for Computational Linguistics. find our card
Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The conll-2015 shared task on shallow discourse parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 1–16, Beijing, China, July.… find our card
Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Bonnie Webber, Attapol Rutherford, Chuan Wang, and Hongmin Wang. 2016. The conll-2016 shared task on multilingual shallow discourse parsing. In Proceedings of the Twentieth Conference on Computational Natural Language Learning - Shared Task, Berlin, Germany,… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rutherford-2016-robust-non-explicit-neural-discourse/

📄 Robust Non-Explicit Neural Discourse Parser in English and Chinese