Elamur Research⌕
☰
home / papers / 2021 / Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition

Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition

Summary

Implicit discourse relation recognition (IDRR) asks a model to label the logical link between two neighbouring sentences when no connective such as "because" or "but" is present. Most earlier systems look only at the sentence pair. The main context-aware alternative, a hierarchical BiLSTM (H-LSTM), mixes the whole paragraph indiscriminately and handles distant dependencies poorly.

The authors propose CT-Net, which converts a paragraph into a paragraph association graph (PAG). In this graph, sentence nodes are joined by adjacency, co-reference and lexical chain edges, and each sentence node connects to its own token nodes. A cross-grained updating mechanism then runs for several rounds. Each round moves information from tokens to sentences (Sentence-state LSTM), across sentences (Multi-Relational GCN), and back to tokens.

The final context vectors for the target pair are fused with local BiLSTM encodings. The model is trained jointly with auxiliary tasks. On PDTB 2.0, CT-Net outperforms models that treat context coarsely. This suggests that selectively tracking relevant context, regardless of distance, benefits IDRR.

Key points

  • Evaluation uses PDTB 2.0, which has 16,224 implicit and 18,459 explicit instances. The split follows Pitler et al. (2009): sections 2-20 for training, 0-1 for validation and 21-22 for test. Tasks are one-vs-others binary and 4-way classification over Comp., Cont., Exp. and Temp., scored by F1 (macro-F1 for 4-way).
  • Each PAG covers 6 sentences, with zero padding for shorter texts. Co-reference chains come from spaCy, and synonyms come from WordNet. A PAG has about 11 edges on average.
  • WordNet-based links cover only about 57–59% of samples in each split.
  • Token nodes start from concatenated char, GloVe and ELMo embeddings. Sentence nodes start as the average of their tokens.
  • Training is multi-task: IDRR is the main objective, supported by two auxiliary objectives, classifying relations that have an overt connective (explicit discourse relation recognition, EDRR) and predicting the connective itself (CP). All three share one encoder but use separate MLPs. The code is public on GitHub (yxuezhang/CTNet).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.naacl-main.126

@inproceedings{zhang2021context,
  title = {{Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition}},
  author = {Zhang, Yingxue and Meng, Fandong and Li, Peng and Jian, Ping and Zhou, Jie},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.126},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.126}
}

References

21 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find our card
  2. Zeyu Dai and Ruihong Huang. 2019. A regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langu… find our card
  3. Fengyu Guo, Ruifang He, Jianwu Dang, and Jr-Yan Wang. 2020. Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition. In AAAI 2020. find
  4. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9:1735– 1780. find
  5. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, Doha, Qatar. Association for Computational Linguistics. find our card
  6. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. International Conference on Learning Representations. find
  7. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1299– 1308, Co… find our card
  8. Wenqiang Lei, Yuanxin Xiang, Yuwei Wang, Qian Zhong, Meichun Liu, and Min-Yen Kan. 2018. Linguistic properties matter for implicit discourse relation recognition: Combining semantic interaction, topic continuity and attribution. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  9. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1224–1233, Austin, Texas. Association for Computational Linguistic… find our card
  10. George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41. find our card
  11. Linh The Nguyen, Linh Van Ngo, Khoat Than, and Thien Huu Nguyen. 2019. Employing the correspondence of relations and connectives to identify implicit discourse relations via label embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4201– 4207… find our card
  12. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  13. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  14. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 683–691, S… find our card
  15. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Asso… find our card
  16. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1006–1017, Vanco… find our card
  17. Samuel Rönnqvist, Niko Schenk, and Christian Chiarcos. 2017. A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 256–262, Vancou… find our card
  18. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pages 593–607. Springer. find
  19. Wei Shi and Vera Demberg. 2019. Next sentence prediction helps implicit discourse relation classification within and across domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EM… find our card
  20. Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, and Guodong Zhou. 2019. Topic tensor network for implicit discourse relation recognition in Chinese. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 608– 618, Florence, Italy. Association for Computation… find our card
  21. Yue Zhang, Qi Liu, and Linfeng Song. 2018. Sentence-state LSTM for text representation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 317–327, Melbourne, Australia. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2021-context-tracking-network-graph-based/