Elamur Research⌕
☰
home / papers / 2022 / CSECU-DSG @ Causal News Corpus 2022: Fusion of RoBERTa Transformers Variants for Causal Event Classification

CSECU-DSG @ Causal News Corpus 2022: Fusion of RoBERTa Transformers Variants for Causal Event Classification

Summary

The paper describes the participation of the CSECU-DSG team in Shared Task 3 of the CASE-2022 workshop. The task was to detect cause-effect relations between events in news texts. The authors worked only on Subtask 1. In it, each sentence must be classified as having causal content or not, i.e., assigned to the class Causal or Non-Causal. The data were taken from the Causal News Corpus (CNC). The system combines two fine-tuned models: the base RoBERTa and Twitter-RoBERTa, which had been pre-trained on tweets. Each model produces class probabilities through a linear layer with softmax. These probabilities are then averaged with weights, and the final label is the class with the highest combined value. The weights were selected by grid search. On the main metric, F1, the system ranked first among the participants. The work shows that even a simple weighted fusion of the scores of two related transformers noticeably improves the result compared with each model alone. This is a useful reference point for automatic extraction of events and causality from news.

Key points

  • Data: CNC (presented at LREC-2022) contains 3559 sentences: 2925 in train, 323 in dev and 311 in test.
  • Models: roberta-base (12 layers, 125M parameters) and cardiffnlp/twitter-roberta-base-sentiment (trained on about 58M tweets). The implementation uses the simpletransformers library. Parameters: learning rate 3e-5, max-len 128, 5 epochs, batch 16.
  • Fusion: the weights were searched in the range from 0.1 to 0.9. The best turned out to be 0.6 for RoBERTa and 0.4 for Twitter-RoBERTa.
  • Result on test: F1 = 0.8619, accuracy = 0.8392, MCC = 0.6714. This is 1st place; the second team (Arguably) had F1 = 0.8610, and the baseline F1 = 0.8120.
  • Ablation: on its own, RoBERTa gives F1 = 0.8516, and Twitter-RoBERTa 0.8245. Fusion raises F1 by about 1 point compared with the first model and by about 4 points compared with the second.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.case-1.19

@inproceedings{aziz2022csecu,
  title = {{CSECU-DSG @ Causal News Corpus 2022: Fusion of RoBERTa Transformers Variants for Causal Event Classification}},
  author = {Aziz, Abdul and Hossain, Md. Akram and Chy, Abu Nowshed},
  booktitle = {Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE)},
  year = {2022},
  doi = {10.18653/v1/2022.case-1.19},
  url = {https://doi.org/10.18653/v1/2022.case-1.19}
}

References

15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul Aziz, Afrin Sultana, Md Akram Hossain, Nabila Ayman, and Abu Nowshed Chy. 2020. Feature fusion with hand-crafted and transfer learning embeddings for cause-effect relation extraction. In FIRE (Working Notes), pages 756–764. find
  2. Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650. find our card
  3. Denis Gordeev, Adis Davletov, Alexey Rey, and Nikolay Arefyev. 2020. Liori at the fincausal 2020 shared task. In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation, pages 45–49. find
  4. Marius Ionescu, Andrei-Marius Avram, George-Andrei Dima, Dumitru-Clementin Cercel, and Mihai Dascalu. 2020. Upb at fincausal-2020, tasks 1 & 2: Causality analysis in financial documents using pre-trained language models. In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and… find
  5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405… find
  6. Paramita Mirza. 2014. Extracting temporal and causal relations between events. In Proceedings of the ACL 2014 Student Research Workshop, pages 10–17. find
  7. T. C. Rajapakse. 2019. Simple Transformers. https://github.com/ThilinaRajapakse/ simpletransformers. ↗
  8. Bryan Rink, Cosmin Adrian Bejan, and Sanda Harabagiu. 2010. Learning textual graph patterns to detect causal event relations. In Twenty-Third International FLAIRS Conference. find
  9. Fiona Anting Tan, Ali Hürriyetoğlu, Tommaso Caselli, Nelleke Oostdijk, Hansi Hettiarachchi, Tadashi Nomoto, Onur Uca, and Farhana Ferdousi Liza. 2022a. Event causality identification with causal news corpus - shared task 3, CASE 2022. In Proceedings of the 5th Workshop on Challenges and Application… find
  10. Fiona Anting Tan, Ali Hürriyetoğlu, Tommaso Caselli, Nelleke Oostdijk, Tadashi Nomoto, Hansi Hettiarachchi, Iqra Ameer, Onur Uca, Farhana Ferdousi Liza, and Tiancheng Hu. 2022b. The causal news corpus: Annotating causal relations in event sentences from news. In Proceedings of the Language Resource… find
  11. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv preprint arXiv:1910.03771. find our card
  12. Jie Yang, Soyeon Caren Han, and Josiah Poon. 2022. A survey on extraction of causal relations from natural language text. Knowledge and Information Systems, pages 1–26. find
  13. Xiaoxiao Yu, Xinzhi Wang, Xiangfeng Luo, and Jianqi Gao. 2022. Multi-scale event causality extraction via simultaneous knowledge-attention and convolutional neural network. Expert Systems, 39(5):e12952. find
  14. Dominique Mariko, Hanna Abi Akl, Estelle Labidurie, Stephane Durfort, Hugues De Mazancourt, and Mahmoud El-Haj. 2020. The financial document causality detection shared task (fincausal 2020). In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisat… find
  15. Brian W Matthews. 1975. Comparison of the predicted and observed secondary structure of t4 phage find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aziz-2022-csecu-dsg-causal-news-corpus/