Elamur Research⌕
☰
home / papers / 2022 / Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition

Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition

Summary

Implicit discourse relation recognition (IDRR) is the task of identifying the sense linking two text arguments when no explicit connective is present. Earlier neural approaches typically relied on a single attention mechanism. The authors argue that different relation senses benefit from different attention types, and that relations between individual words carry additional useful signal. They propose MANF (Multi-Attentive Neural Fusion), which has two parts. The Dual Attention Network (DAN) runs a BiLSTM over word2vec inputs, with BERT tested as an alternative. It builds self-attentive and interactive-attentive representations of each argument and merges them through a learned fusion gate. The Offset Matrix Network (OMN) computes embedding differences for every cross-argument word pair, weights them with attention, and passes the result through an MLP to produce "linguistic evidence." A second gate combines both representations before softmax classification. On the four top-level senses of PDTB 3.0, MANF is reported to achieve state-of-the-art performance, which also gives a new baseline on this newer corpus. Attention visualizations suggest that the two attention types focus on different kinds of words.

Key points

  • Architecture: MANF combines DAN (semantic connection) and OMN (word-pair offset evidence) through a hierarchical, two-level gated fusion. It is trained with cross-entropy loss plus L2 regularization, using the Adam optimizer and dropout.
  • Data: The experiments use PDTB 3.0 with the standard PDTB 2.0 split: sections 2–20 for training, 0–1 for development, and 21–22 for testing.
  • Dataset size: The split contains 17,945 training, 1,653 development and 1,474 test implicit instances. These cover four senses: Expansion, Comparison, Contingency and Temporal.
  • Attention behaviour:
    • Interactive attention tends to emphasize function words that may signal how the arguments connect.
    • Self-attention favours content words.
    • The fusion gate lets the model weight each type differently depending on the sense.
  • OMN input choice: Using raw pre-trained embeddings as OMN input, rather than BiLSTM states, gives a view that complements DAN. Baselines compared include NNMA, ANN and IPAL.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-acl.256

@inproceedings{xiang2022encoding,
  title = {{Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition}},
  author = {Xiang, Wei and Wang, Bang and Dai, Lu and Mo, Yijun},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.256},
  url = {https://doi.org/10.18653/v1/2022.findings-acl.256}
}

References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hongxiao Bai and Hai Zhao. 2018. Deep Enhanced Representation for Implicit Discourse Relation Recognition. In Proceedings of the 27th International Conference on Computational Linguistics, pages 571–583, Stroudsburg, PA. The Association for Computational Linguistics. find our card
  2. Or Biran and Kathleen McKeown. 2013. Aggregated Word Pair Features for Implicit Discourse Relation Disambiguation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 69–73, Stroudsburg, PA. The Association for Computational Linguistics. find our card
  3. Sasha Blair-Goldensohn, Kathleen McKeown, and Owen Rambow. 2007. Building and Refining Rhetorical-Semantic Relation Models. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 428–435, Stroudsburg, PA. The Association for Compu… find
  4. Jifan Chen, Qi Zhang, Pengfei Liu, and Xuanjing Huang. 2016a. Discourse relations detection via a mixed generative-discriminative framework. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 2921–2927, Menlo Park, Calif. AAAI Press. find
  5. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuan-Jing Huang. 2016b. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1726–1735, Stroudsburg, PA. The Ass… find our card
  6. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167, New York, NY. ACMPress. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association Computational Linguistics, pages 4171–4186, Stroudsburg, PA… find
  8. Fengyu Guo, Ruifang He, and Jianwu Dang. 2019. Implicit discourse relation recognition via a bilstm-cnn architecture with dynamic chunk-based max pooling. IEEE Access, 7:169281–169292. find
  9. Fengyu Guo, Ruifang He, Jianwu Dang, and Jian Wang. 2020. Working Memory-Driven Neural Networks with a Novel Knowledge Enhancement Paradigm for Implicit Discourse Relation Recognition. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, pages 7822–7829, Menlo Park, Calif.… find
  10. Fengyu Guo, Ruifang He, Di Jin, Jianwu Dang, Long-biao Wang, and Xiangang Li. 2018. Implicit discourse relation recognition using neural tensor network with interactive attention and sparse learning. In Proceedings of the 27th International Conference on Computational Linguistics, pages 547– 558, St… find our card
  11. Francisco Guzmán, Shafiq Joty, Lluís Màrquez, and Preslav Nakov. 2014. Using discourse structure improves machine translation evaluation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 687– 698, Stroudsburg, PA. The Association for Computational Lin… find
  12. Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, and Guillaume R Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in neural information processing systems, pages 3167–3175, Cambridge, MA. MIT Press. find
  13. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344. find our card
  14. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1299– 1308, St… find our card
  15. Maria Liakata, Simon Dobnik, Shyamasree Saha, Colin Batchelor, and Dietrich Rebholz Schuhmann. 2013. A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Pro… find
  16. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing Implicit Discourse Relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 343–351, Stroudsburg, PA. The Association for Computer Linguistics. find our card
  17. Yang Liu and Sujian Li. 2016. Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1224– 1233, Stroudsburg, PA. The Association for Computational Lin… find our card
  18. Annie Louis, Aravind Joshi, Rashmi Prasad, and Ani Nenkova. 2010. Using entity features to classify implicit discourse relations. In Proceedings of the SIG-DIAL 2010 Conference, pages 59–62, Stroudsburg, PA. The Association for Computer Linguistics. find our card
  19. Easily Identifiable Discourse Relations. In Proceedings of the 22th International Conference on Computational Linguistics, pages 87–90, Stroudsburg, PA. The Association for Computer Linguistics. find
  20. Huibin Ruan, Yu Hong, Yang Xu, Zhen Huang, Guodong Zhou, and Min Zhang. 2020. Interactively-Propagative Attention Learning for Implicit Discourse Relation Recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3168–3178, Stroudsburg, PA. The Association… find our card
  21. Attapol Rutherford, Vera Demberg, and Nianwen Xue. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 281–291, Stroudsburg, PA. The Association for… find our card
  22. Chang Wang and Bang Wang. 2020. An End-to-end Topic-Enhanced Self-Attention Network for Social Emotion Classification. In Proceedings of The Web Conference 2020, pages 2210–2219, New York, NY. ACMPress. find
  23. Wei Xiang and Bang Wang. 2019. A survey of event extraction from text. IEEE Access, 7:173111–173137. find
  24. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive Attention Networks for Aspect-Level Sentiment Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 4068–4074, Amsterdam, Netherlands. Elsevier. find
  25. Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, and Guodong Zhou. 2019. Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 608– 618, Stroudsburg, PA. The Association for Computa… find our card
  26. Fandong Meng, Zhengdong Lu, Hang Li, and Qun Liu. 2016. Interactive Attention for Neural Machine Translation. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers, pages 2174–2185, Stroudsburg, PA. The Association for Computational Linguistics. find
  27. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2230–2235, Stroudsburg, PA. The Associat… find our card
  28. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, Cambridge, MA. MIT Press. find
  29. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pa… find our card
  30. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind K. Joshi. 2008. find
  31. Yingxue Zhang, Fandong Meng, Li Peng, Jian Ping, and Jie Zhou. 2021. Context tracking network: Graph-based context modeling for implicit discourse relation recognition. In Proceedings of the 2021 Conference of the North American Chapter of the Association Computational Linguistics, pages 1592–1599,… find our card
  32. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Stroudsburg, PA. The Association… find our card
  33. Xiaopei Zhou, Yu Hong, Tingting Che, Jianmin Yao, and Qiaoming Zhu. 2013. An unsupervised approach to inferring implicit discourse relation. Journal of Chinese Information Processing, 27(2):17– 25. find

↑ back to top