Elamur Research⌕
☰
home / papers / 2021 / Does It Happen? Multi-hop Path Structures for Event Factuality Prediction with Graph Transformer Networks

Does It Happen? Multi-hop Path Structures for Event Factuality Prediction with Graph Transformer Networks

Paper · 2021 · Duong Le and Thien Huu Nguyen

Summary

Event Factuality Prediction (EFP) estimates how certain a text is that an event mention really took place, as a real-valued score from -3 (surely did not happen) to +3 (surely happened). Earlier neural EFP systems built sentence graphs only from direct links: single dependency edges or pairwise contextual similarity. The authors argue that the cue words deciding factuality often sit several steps away from the trigger. These words can only be reached through chains that mix syntactic and semantic links. Their model starts from BERT base word vectors and builds syntactic and semantic importance matrices that are conditioned on the trigger. A Graph Transformer Network mixes these matrices with learned weights and multiplies the mixtures, so that multi-hop, mixed-type paths emerge. A GCN then encodes the words over the learned structures. EFP training sets are small, so the authors add an information-bottleneck regularizer that limits the mutual information between the GTN output and the input encoding. The model reports state-of-the-art results on four benchmark datasets, and ablations show that every component contributes.

Key points

  • EFP is framed as regression over [-3, 3]. In the example "He cannot go to the restaurant", the trigger "go" gets -3.
  • Words are encoded with BERT base, taking the first wordpiece from the last layer (768 dimensions). This matches the setup of Veyseh et al. (2019a), the previous best system, for a fair comparison.
  • The syntactic structure is customized to the trigger using shortest-path distance in the dependency tree. Words closer to the trigger receive higher importance.
  • GTN (Yun et al., 2019) forms intermediate structures as weighted sums of the initial matrices and multiplies them to model multi-hop paths. The information bottleneck adds a mutual-information loss term, L_disc, in the style of Belghazi et al. (2018).
  • The ablations are "- GTN", "- Multi-Hop", "- IB" and "- IB + BERT in R", and all of them reduce performance. For the ablated variants the authors re-tuned the settings to M=3, C=2, G=2. Error analysis points to unknown words as a remaining difficulty.

Interesting ideas

  • I1. The paper weights the context graph by how far each word is from the target predicate in the dependency tree, so nearby dependents count more than distant words.
  • I2. Factuality cues can be reached only by chaining different edge types, such as a semantic link (family members ↔ brother) followed by dependency edges up to "confirmed".
  • I3. Factuality is a graded score rather than a yes/no label, which lets a system separate "probably" from "certainly".
  • I4. The "- IB + BERT in R" ablation removes the regularizer but gives the predictor the raw encoder features, which tests whether the gain comes from the method or just from extra input.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): when assigning modality or factuality to a predicate frame, rank candidate cue words (negation, modals, reporting verbs) by UD path distance to the predicate, and log that distance as part of the explanation.
  • I2 (world): before committing an event to the world state as happened, follow mixed paths of coreference and dependency edges to the governing attitude verb, and keep that path as the human-readable justification.
  • I3 (world): store a signed certainty value on each event node, so that answers to "did X happen?" questions can be hedged instead of being forced into true or false.
  • I4 (learn): add such feature-matched controls to evaluations, so that an improvement from a new module is not mistaken for a side effect of extra information.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OregonEugene · United States14 works · 3,167 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Duong Le and Thien Huu Nguyen. 2021. Does It Happen? Multi-hop Path Structures for Event Factuality Prediction with Graph Transformer Networks. Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.wnut-1.6

@inproceedings{le2021does,
  title = {{Does It Happen? Multi-hop Path Structures for Event Factuality Prediction with Graph Transformer Networks}},
  author = {Duong Le and Thien Huu Nguyen},
  booktitle = {Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)},
  year = {2021},
  doi = {10.18653/v1/2021.wnut-1.6},
  url = {https://doi.org/10.18653/v1/2021.wnut-1.6}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Duong Minh Le, My Thai, and Thien Huu Nguyen. 2020. Multi-task learning for metaphor detection with graph convolutional neural networks and word sense disambiguation. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI). find
  2. Heike Adel and Hinrich Schütze. 2017. Exploring different dimensions of attention for uncertainty detection. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL). ↗
  3. Kenton Lee, Yoav Artzi, Yejin Choi, and Luke Zettlemoyer. 2015. Event detection and factuality assessment with non-expert supervision. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  4. Rachel Rudinger, Aaron Steven White, and Benjamin Van Durme. 2018. Neural models of factuality. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).
  5. Amnon Lotan, Asher Stern, and Ido Dagan. 2013. Truthteller: Annotating predicate truth. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
  6. A-L Minard, Manuela Speranza, Ruben Urizar, Begona Altuna, MGJ van Erp, AM Schoen, CM van Son, et al. 2016. Meantime, the newsreader multilingual event and time corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC). find
  7. Roser Saurí. 2008. A factuality profiler for eventualities in text. Unveröffentlichte Dissertation, Brandeis University. Zugriff auf http://www.cs. brandeis. edu/˜ roser/pubs/sauriDiss, 1. ↗
  8. Roser Saurí and James Pustejovsky. 2009. Factbank: a corpus annotated with event factuality. Language resources and evaluation, 43(3):227. ↗
  9. Teruko Mitamura, Zhengzhong Liu, and Eduard Hovy. 2015. Overview of tac kbp 2015 event nugget track. In Proceedings of Text Analysis Conference (TAC). find
  10. Roser Saurí and James Pustejovsky. 2012. Are you sure that this happened? assessing the factuality degree of events in text. Computational Linguistics, 38(2):261–299. ↗
  11. Rowan Nairn, Cleo Condoravdi, and Lauri Karttunen. 2006. Computing relative polarity for textual inference. In Proceedings of the fifth international workshop on inference in computational semantics (icos-5).
  12. Gabriel Stanovsky, Judith Eckle-Kohler, Yevgeniy Puzikov, Ido Dagan, and Iryna Gurevych. 2017. Integrating deep linguistic features in factuality prediction over unified datasets. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  13. Minh Van Nguyen, Viet Dac Lai, and Thien Huu Nguyen. 2021. Cross-task instance representation interactions and label dependencies for joint information extraction with graph convolutional networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
  14. Naftali Tishby, Fernando C Pereira, and William Bialek. 2000. The information bottleneck method. In arXiv preprint physics/0004057. ↗
  15. Amir Pouran Ben Veyseh, Viet Lai, Franck Dernoncourt, and Thien Huu Nguyen. 2021. Unleash GPT-2 power for event detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP). find
  16. Thien Huu Nguyen and Ralph Grishman. 2015. Event detection and domain adaptation with convolutional neural networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  17. Thien Huu Nguyen and Ralph Grishman. 2018. Graph convolutional networks with argument-aware pooling for event detection. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI). find
  18. Amir Pouran Ben Veyseh, Thien Huu Nguyen, and Dejing Dou. 2019a. Graph based neural networks for event factuality prediction using syntactic and semantic structures. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  19. Thien Huu Nguyen, Adam Meyers, and Ralph Grishman. 2016. New york university 2016 system for kbp event nugget: A deep learning approach. In Proceedings of Text Analysis Conference (TAC). find
  20. Amir Pouran Ben Veyseh, My T. Thai, Thien Huu Nguyen, and Dejing Dou. 2019b. Rumor detection in social networks via deep contextual modeling. In Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). find
  21. Vinodkumar Prabhakaran, Owen Rambow, and Mona Diab. 2010. Automatic committed belief tagging. In Proceedings of the International Conference on Computational Linguistics (COLING).
  22. Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. 2019. Graph transformer networks. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). ↗
  23. Zhong Qian, Peifeng Li, Yue Zhang, Guodong Zhou, and Qiaoming Zhu. 2018. Event factuality identification via generative adversarial networks with auxiliary classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJ-CAI). find
  24. Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  25. Zhong Qian, Peifeng Li, and Qiaoming Zhu. 2015. A two-step approach for event factuality identification. In Asian Language Processing (IALP), 2015 International Conference on, pages 103–106. IEEE. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/le-2021-does-it-happen-multi-hop/