Elamur Research⌕
☰
home / papers / 2017 / Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification

Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification

Summary

The paper addresses implicit discourse relation classification. In this task, two adjacent arguments carry a relation (e.g., Cause) without an overt connective such as "because". The authors note that explicit relations are easy to classify (over 85% accuracy), while implicit ones stay below roughly 50%. They therefore exploit the implicit connectives that annotators inserted into the Penn Discourse Treebank (PDTB 2.0), which are available only during training.

They build two CNN-based recognizers:

  • a-CNN sees the arguments plus the annotated connective.
  • i-CNN sees only the raw arguments.

Instead of using a fixed distance measure, a discriminator tries to tell the two networks' hidden features apart. i-CNN learns to classify relations while also fooling the discriminator. This pushes it to produce connective-like representations without ever seeing connectives at test time.

Evaluated on PDTB in multi-class and one-versus-all settings, the approach outperforms standalone neural models and earlier connective-based methods. The authors report state-of-the-art results. The work matters because it shows a general way to transfer training-only side information through feature-level adversarial imitation.

Key points

  • Architecture: an implicit network (i-CNN), a connective-augmented network (a-CNN), a shared relation classifier, and a feature discriminator. Training is end-to-end with back-propagation, and the adversarial game is described as the first in discourse parsing.
  • One-versus-all F1 on PDTB (train sections 2–20, dev 21–22, test 0–1):
    • Comparison: 40.87
    • Contingency: 54.56
    • Expansion: 72.38, the best among the compared systems
    • Temporal: 36.20
  • The method clearly beats Zhou et al. (2010), a pipeline that first predicts implicit connectives. It is also reported as more effective than a fixed ℓ2 feature-matching regularizer.
  • Training dynamics: discriminator accuracy rises above 0.9 early, then drops to about 0.6 as i-CNN learns to confuse it. In t-SNE plots, the features of the two networks become increasingly mixed.
  • Multi-class experiments use the PDTB-Lin and PDTB-Ji settings. The authors relate the approach to knowledge distillation but note it imitates features rather than predictions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

57 papers of our library cite it (of the 140673 whose bibliographies we read); 26 of them have a page here, 31 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1093 · arXiv 1704.00217

@inproceedings{qin2017adversarial,
  title = {{Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification}},
  author = {Qin, Lianhui and Zhang, Zhisong and Zhao, Hai and Hu, Zhiting and Xing, Eric},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1093},
  eprint = {1704.00217},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p17-1093}
}

References

46 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Or Biran and Kathleen McKeown. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL, Volume 2: Short Papers). Sofia, Bulgaria, pages 69–73. find our card
  2. Chloé Braud and Pascal Denis. 2015. Comparing word representations for implicit discourse relation classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Lisbon, Portugal, pages 2201–2211. find our card
  3. Chloé Braud and Pascal Denis. 2016. Learning connective-based word representations for implicit discourse relation identification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, Texas, pages 203–213. find our card
  4. Deng Cai, Hai Zhao, Zhisong Zhang, Yuan Xin, Yongjian Wu, and Feiyue Huang. 2017. Fast and accurate neural word segmentation for Chinese. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). Vancouver, Canada. find
  5. Changge Chen, Peilu Wang, and Hai Zhao. 2015. Shallow discourse parsing using constituent parsing tree. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task (CONLL). Beijing, China, pages 37– 41. find our card
  6. Changge Chen and Hai Zhao. 2015. Deceptive opinion spam detection using deep level linguistic feature. In The 4th CCF Conference on Natural Language Processing and Chinese Computing (NLPCC 2015), LNCS. Nanchang, China, volume 9362, pages 465– 474. find
  7. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016a. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL Volume 1: Long Papers). Berlin, German… find our card
  8. Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016b. In-fogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems. pages 2172–2180. find
  9. Xilun Chen, Ben Athiwaratkun, Yu Sun, Kilian Weinberger, and Claire Cardie. 2016c. Adversarial deep averaging networks for cross-lingual sentiment classification. arXiv preprint arXiv:1606.01614 . find
  10. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12(Jul):2121–2159. find
  11. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Lavi-olette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. Journal of Machine Learning Research 17(59):1–35. find
  12. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, T. Raymond Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 1602–1613. find
  13. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. pages 2672–2680. find
  14. Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric P Xing. 2016a. Harnessing deep neural networks with logic rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany, pages 2410–2420. find
  15. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Controllable text generation. arXiv preprint arXiv:1703.00955 . find our card
  16. Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, and Eric P Xing. 2016b. Deep neural networks with massive learned knowledge. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, USA, pages 1670–1679. find
  17. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics (TACL) 3:329– 344. find our card
  18. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse-driven language models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL). San… find
  19. Yangfeng Ji, Gongbo Zhang, and Jacob Eisenstein. 2015. Closing the gap: Domain adaptation from explicit to implicit discourse relations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Lisbon, Portugal, pages 2219–2224. find our card
  20. Alex M Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016. Professor forcing: A new algorithm for training recurrent networks. In Advances In Neural Information Processing Systems. pages 4601–4609. find
  21. Man Lan, Yu Xu, and Zhengyu Niu. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL, Volume 1: Long Papers). Sofia, Bulgaria, pages 476–485. find our card
  22. Junyi Jessy Li, Marine Carpuat, and Ani Nenkova. 2014. Assessing the discourse factors that influence the quality of machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL, Volume 2: Short Papers). Baltimore, Maryland, pages 283–288. find
  23. Yujia Li, Kevin Swersky, and Richard S Zemel. 2015. Generative moment matching networks. In Proceedings of the 32nd International Conference on Machine Learning (ICML). Lille, France, pages 1718–1727. find
  24. Zhongyi Li, Hai Zhao, Chenxi Pang, Lili Wang, and Huan Wang. 2016. A constituent syntactic parse tree based discourse parser. In Proceedings of the Twentieth Conference on Computational Natural Language Learning - Shared Task (CONLL). Berlin, Germany, pages 60–64. find our card
  25. Maria Liakata, Simon Dobnik, Shyamasree Saha, Colin Batchelor, and Dietrich Rebholz-Schuhmann. 2013. A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Pro… find
  26. Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, and Eric P Xing. 2017. Recurrent topic-transition GAN for visual paragraph generation. arXiv preprint arXiv:1703.07022 . — [arXiv:1703.07022](https://arxiv.org/abs/1703.07022) find
  27. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP). Singapore, pages 343– 351. find our card
  28. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, Texas, pages 1224–1233. find our card
  29. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. arXiv preprint arXiv:1603.02776 . find
  30. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9:2579–2605. find
  31. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (3). South Lake Tahoe, Nevada, USA, pages 3111–3119. find
  32. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of he Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Proc… find our card
  33. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The Penn discourse treebank 2.0. In The sixth international conference on Language Resources and Evaluation (LREC). Marrakech, Morocco, pages 2961–2968. find our card
  34. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016a. Implicit discourse relation recognition with context-aware character-enhanced embeddings. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan, pages 1914–1924. find our card
  35. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016b. Shallow discourse parsing using convolutional neural network. In Proceedings of the CoNLL-16 shared task. Berlin, Germany, pages 70–77. find our card
  36. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016c. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, Texas, pages 2263–2270. find our card
  37. Attapol Rutherford and Nianwen Xue. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL:… find our card
  38. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems. pages 2226–2234. find our card
  39. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Highway networks. arXiv preprint arXiv:1505.00387 . find our card
  40. Peilu Wang, Yao Qian, Frank K. Soong, Lei He, and Hai Zhao. 2016. Learning distributed word representations for bidirectional LSTM recurrent neural network. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi… find
  41. Yu Xu, Man Lan, Yue Lu, Zheng Yu Niu, and Chew Lim Tan. 2012. Connective prediction using machine learning for implicit discourse relation classification. In The 2012 International Joint Conference on Neural Networks (IJCNN). Brisbane, Australia, pages 1–8. find
  42. Biao Zhang, Deyi Xiong, jinsong su, Qun Liu, Ron-grong Ji, Hong Duan, and Min Zhang. 2016a. Variational neural discourse relation recognizer. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, Texas, pages 382–391. find our card
  43. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task (CoNLL). Beijing, China, pages 1–16… find our card
  44. Zhisong Zhang, Hai Zhao, and Lianhui Qin. 2016b. Probabilistic graph-based dependency parsing with convolutional neural network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany, pages 1382–1392. find
  45. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Bonnie Webber, Attapol Rutherford, Chuan Wang, and Hongmin Wang. 2016. The CoNLL-2016 shared task on shallow discourse parsing. In Proceedings of the Twentieth Conference on Computational Natural Language Learning - Shared Task (CoNLL). Berlin, Germany, page… find
  46. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics (CoLING 2010). Beijing, China, pages 1507–1514. find our card

↑ back to top