Elamur Research⌕
☰
home / papers / 2020 / Event Extraction as Multi-turn Question Answering

Event Extraction as Multi-turn Question Answering

Summary

The paper reframes event extraction (finding triggers, their event types, and role-labelled arguments) as a sequence of reading-comprehension questions instead of a classification task. Classification-based systems treat event types and argument roles as fixed labels. As a result, they ignore what those labels mean, miss how arguments depend on one another, and cannot handle unseen types without new annotation. The authors propose MQAEE, a pipelined BERT-based framework with three stages:

  • trigger identification as extractive QA;
  • trigger classification as YES/NO questions pairing a candidate trigger with an event type;
  • argument extraction as successive extractive questions, one per role, built from the predicted type.

Spans are tagged with BIO labels, so one question can yield several answers. A history answer embedding feeds earlier answers into later turns. The system does not rely on golden entities. On ACE 2005, MQAEE sets a new state of the art for argument extraction. In few-shot tests, pretraining on frequent types and fine-tuning on a handful of examples gives usable results on unseen event types.

Key points

  • Evaluated on ACE 2005, MQAEE reaches 53.4% argument extraction F1, which is 2.0 points above the previous best. Unlike most earlier systems, it uses no golden entity mentions.
  • Modelling question history across turns improves argument classification F1 by 3.0 points over a single-turn variant (QAEE). A case study shows that knowing one argument, such as the Person, helps the model recover a related one, such as the Entity.
  • Generalization was tested with 13-way 1-shot and 5-shot learning after pretraining on the top M = 5, 10 or 20 event types. With M = 20 and 5 shots, argument classification F1 is 47.7, versus 61.1 for the model trained on all 33 types. Performance rises as M grows.
  • The question templates encode the event type plus the trigger with its position, and the role name. This lets new types and roles be expressed as questions rather than as new labels.
  • The authors plan to use reinforcement learning to choose the order in which argument roles are queried.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

77 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 71 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.findings-emnlp.73

@inproceedings{li2020event,
  title = {{Event Extraction as Multi-turn Question Answering}},
  author = {Li, Fayuan and Peng, Weihua and Chen, Yuguang and Wang, Quan and Pan, Lu and Lyu, Yajuan and Zhu, Yong},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.73},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.73}
}

References

34 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proces… find our card
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  4. Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Güney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine. CoRR, abs/1704.05179. find our card
  5. Ralph Grishman, David Westbrook, and Adam Meyers. 2005. Nyu’s english ace 2005 system description. ACE, 5. find
  6. Yu Hong, Jianfeng Zhang, Bin Ma, Jianmin Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using cross-entity inference to improve event extraction. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1127–1136, Portland, Oregon, US… find our card
  7. Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Sebastian Riedel, and Clare Voss. 2018. Zero-shot transfer learning for event extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2160–2170, Melbourne, Australia. Associat… find our card
  8. Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. In Proceedings of ACL-08: HLT, pages 254–262, Columbus, Ohio. Association for Computational Linguistics. reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learni… find our card
  9. Qi Li, Heng Ji, and Liang Huang. 2013. Joint event extraction via structured prediction with global features. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 73–82, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  10. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1340–1350, Florence, Italy. Association fo… find our card
  11. Shasha Liao and Ralph Grishman. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 789–797, Uppsala, Sweden. Association for Computational Linguistics. find our card
  12. Shulin Liu, Kang Liu, Shizhu He, and Jun Zhao. 2016. A probabilistic soft logic based approach to exploiting latent and global information in event classification. In AAAI. find
  13. Xiao Liu, Zhunchen Luo, and Heyan Huang. 2018. Jointly multiple events extraction via attention-based graph information aggregation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1247–1256, Brussels, Belgium. Association for Computational Linguistic… find our card
  14. Wei Lu and Dan Roth. 2012. Automatic event extraction with structured preference modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 835–844, Jeju Island, Korea. Association for Computational Linguistics. find
  15. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. CoRR, abs/1806.08730. find our card
  16. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. CoRR, abs/1705.03551. find our card
  17. Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grishman. 2016. Joint event extraction via recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 300–309, San Diego, California. As… find our card
  18. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of th… find
  19. Chen Qu, Liu Yang, Minghui Qiu, W. Bruce Croft, Yongfeng Zhang, and Mohit Iyyer. 2019. Bert with history answer embedding for conversational question answering. In SIGIR. find
  20. Shuohang Wang and Jing Jiang. 2016. Machine comprehension using match-lstm and answer pointer. CoRR, abs/1608.07905. find our card
  21. Shuohang Wang, Mo Yu, Jing Jiang, Wei Zhang, Xiaoxiao Guo, Shiyu Chang, Zhiguo Wang, Tim Klinger, Gerald Tesauro, and Murray Campbell. 2017a. Evidence aggregation for answer reranking in open-domain question answering. CoRR, abs/1711.05116. — [arXiv:1711.05116](https://arxiv.org/abs/1711.05116) find
  22. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. find our card
  23. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017b. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 189–198, Vancouver, Canada. As… find our card
  24. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. ArXiv, abs/1611.01603. find our card
  25. Caiming Xiong, Victor Zhong, and Richard Socher. 2018. DCN+: Mixed objective and deep residual coattention for question answering. In International Conference on Learning Representations. find
  26. Lei Sha, Jing Liu, Chin-Yew Lin, Sujian Li, Baobao Chang, and Zhifang Sui. 2016. RBPB: Regularization-based pattern balancing method for event extraction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1224– 1234, Berlin, Ger… find
  27. Bishan Yang and Tom M. Mitchell. 2016. Joint extraction of events and entities within a document context. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 289–299, San Diego, California. Associati… find our card
  28. Lei Sha, Feng Qian, Baobao Chang, and Zhifang Sui. 2018. Jointly extracting event triggers and arguments by dependency-bridge rnn and tensor-based argument interaction. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  29. Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan, and Dongsheng Li. 2019. Exploring pre-trained language models for event extraction and generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5284–5294, Florence, Italy. Association for Computationa… find our card
  30. Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. 2016. Reasonet: Learning to stop reading in machine comprehension. CoRR, abs/1609.05284. — [arXiv:1609.05284](https://arxiv.org/abs/1609.05284) find
  31. Chuanqi Tan, Furu Wei, Nan Yang, Weifeng Lv, and Ming Zhou. 2017. S-net: From answer extraction to answer generation for machine reading comprehension. CoRR, abs/1706.04815. — [arXiv:1706.04815](https://arxiv.org/abs/1706.04815) find
  32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. find
  33. David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. find our card
  34. Chuan Wang, Nianwen Xue, and Sameer Pradhan. 2015. Boosting transition-based amr parsing with refined actions and auxiliary analyzers. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2020-event-extraction-as-multi-turn/