Elamur Research⌕
☰
home / papers / 2019 / Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

Summary

Implicit discourse relations, where two adjacent text units are linked without an explicit connective, are much more frequent in Chinese than in English. Chinese is paratactic and often omits connectives. Most existing models were built for English and rely only on sentence-level argument representations, which the authors argue carry too little information for Chinese.

They propose the Topic Tensor Network (TTN), which combines two views of each argument:

  • a sentence-level encoding from a GCN (Gated Convolutional Network);
  • a topic-level encoding from a Simplified Topic Model (STM), a VAE-style neural topic model that infers a latent topic distribution from the bag-of-words input.

Each pair of representations goes into its own Factored Tensor Network (FTN). The FTN uses multi-slice tensors with factorization to model interactions between the arguments at lower cost. An MLP classifier then predicts the relation.

On the Chinese corpus CDTB, TTN outperforms a number of leading prior models on micro F1 as well as macro F1. This shows that topic continuity is a useful signal for Chinese discourse parsing.

Key points

  • Implicit relations make up 75.2% of CDTB (500 newswire documents) but only about 40% of the English PDTB. The relation classes are Causality, Coordination, Elaboration and Transition.
  • Adding the FTNs raises F1 for Elaboration by 3.9 points and for Transition by 8.0 points.
  • Factorization cuts tensor parameters about 20-fold compared with the NTN of Guo et al. (2018). Substituting that NTN lowers micro F1 by 1.0 and macro F1 by 1.4.
  • Most errors involve the majority class Coordination, which is 61.6% of the training data. For example, 51.3% of Causality samples are mislabeled as Coordination.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1058

@inproceedings{xu2019topic,
  title = {{Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese}},
  author = {Xu, Sheng and Li, Peifeng and Kong, Fang and Zhu, Qiaoming and Zhou, Guodong},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1058},
  url = {https://doi.org/10.18653/v1/p19-1058}
}

References

31 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. In Proceedings of the 27th International Conference on Computational Linguistics (COL-ING), pages 571–583. find our card
  2. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114. find our card
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, pages 85–112. find our card
  4. Fang Kong and Guodong Zhou. 2017. A CDT-styled end-to-end Chinese discourse parser. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):26. find
  5. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1726–1735. find our card
  6. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1299–1… find our card
  7. National Institute of Child Health and Human Development. 2000. Teaching children to read: An Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The Penn Discourse Tree-Bank 2.0. In The 6th international conference on Language Resour… find
  8. Wenqiang Lei, Yuanxin Xiang, Yuwei Wang, Qian Zhong, Meichun Liu, and Min-Yen Kan. 2018. Linguistic properties matter for implicit discourse relation recognition: Combining semantic interaction, topic continuity and attribution. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (… find
  9. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2263–2270. find our card
  10. Yancui Li, Fang Kong, and Guodong Zhou. 2014. Building Chinese discourse corpus with connective-driven dependency tree structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2105– 2114. find
  11. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1006–1017. find our card
  12. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 343–351. find our card
  13. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning (ICML), pages 1278–1286. find our card
  14. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1224– 1233. find our card
  15. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI), pages 2750–2756. find
  16. Samuel Rönnqvist, Niko Schenk, and Christian Chiarcos. 2017. A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pages 256–262. find our card
  17. Yang Liu, Jiajun Zhang, and Chengqing Zong. 2017. Memory augmented attention model for Chinese implicit discourse relation recognition. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data (CCL), pages 411–423. find
  18. Wei Shi, Frances Yung, Raphael Rubino, and Vera Demberg. 2017. Using explicit discourse connectives in translation for implicit discourse relation classification. In Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP), pages 484–495. find our card
  19. Yishu Miao, Lei Yu, and Phil Blunsom. 2016. Neural variational inference for text processing. In Proceedings of the 33rd International Conference on Machine Learning (ICML), pages 1727–1736. find
  20. Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in neural information processing systems (NIPS), pages 926–934. find
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS), pages 3111–3119. find
  22. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pages 184–188. find our card
  23. Wenzhe Pei, Tao Ge, and Baobao Chang. 2014. Max-margin tensor neural network for Chinese word segmentation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 293–303. find
  24. Yang Xu, Yu Hong, Huibin Ruan, Jianmin Yao, Min Zhang, and Guodong Zhou. 2018. Using active learning to expand training data for implicit discourse relation recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 725–731. find our card
  25. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-IJCNLP), pag… find our card
  26. Naiwen Xue, Fei Xia, Fudong Chiou, and Martha Palmer. 2005. The Penn Chinese TreeBank: Phrase structure annotation of a large corpus. Natural Language Engineering, 11(2):207–238. find
  27. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Bonnie Webber, Attapol Rutherford, Chuan Wang, and Hongmin Wang. 2016. The CoNLL-2016 shared task on shallow discourse parsing. In Proceedings of the 20th Conference on Computational Natural Language Learning - Shared Task (CoNLL), pages 1–19. find
  28. Jichuan Zeng, Jing Li, Yan Song, Cuiyun Gao, Michael R Lyu, and Irwin King. 2018. Topic memory networks for short text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3120–3131. find
  29. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2230–2235. find our card
  30. Yuping Zhou and Nianwen Xue. 2015. The Chinese Discourse TreeBank: a Chinese corpus annotated with discourse relations. Language Resources and Evaluation, 49(2):397–431. find
  31. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pages 1507–1514. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2019-topic-tensor-network-implicit-discourse/