Elamur Research⌕
☰
home / papers / 2026 / Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

Paper · 2026 · Abdullah Al Monsur, Nitesh Vamshi Bommisetty and Gene Louis Kim

Summary

The paper examines two persistent weaknesses in event detection (ED) research. First, decoder-only LLMs attend only to preceding tokens, which limits the token embeddings they produce for understanding tasks. Second, the field mostly reports Micro-F1, which rewards strong performance on frequent event types and hides failures on rare ones.

Following the MAVEN task definition (trigger identification plus trigger classification), the authors use Llama and Qwen as embedding backbones inside a word-level classifier. They add sentence-level context through several variants: BaseTE, ConcatPool, Gating, FiLM and BiSE-BiLSTM. They compare these against encoder models (BERT, T5, RoBERTa, DeBERTa-v3), against the bidirectional adaptations Dec2Enc and LLM2Vec, and against zero-shot and few-shot prompting. Experiments cover MAVEN and RAMS, with and without LoRA.

Adding sentence context helps the decoder-only models. FiLM is the strongest variant overall. With these adaptations, Llama and Qwen outperform BERT and roughly match RoBERTa. LoRA behaves like a regularizer and raises Macro-F1 in nearly every setup. The work argues that Macro-F1 should be reported routinely, because model rankings change depending on which metric is used.

Key points

  • Datasets: MAVEN (built from English Wikipedia) and RAMS (built from news articles). Both have many event types with a long-tailed frequency distribution.
  • Prompting baselines on MAVEN:
    • Zero-shot prompting performs poorly. Llama 3 (8B) reaches 36.10% Micro-F1 but only 24.11% Macro-F1.
    • The best few-shot setting for the same model reaches 63.95% Micro-F1 and 54.13% Macro-F1.
  • Statistical significance: A paired t-test on Macro-F1 shows the Llama and Qwen variants differ significantly (p<0.05) from BERT and T5. Some variants are not significantly different from RoBERTa.
  • LoRA on TextEE's DyGIEpp (RoBERTa): Micro-F1 is essentially unchanged (65.33 vs 65.39). Precision rises from 62.47 to 64.92, while recall falls from 68.46 to 64.83.
  • Limitations noted by the authors:
    • Only one data split was used.
    • The hyperparameter search was limited.
    • There was no unified comparison against current state-of-the-art ED systems.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of South FloridaTampa · United States7 works · 33,127 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 167 works

Cite

Cite as Abdullah Al Monsur, Nitesh Vamshi Bommisetty and Gene Louis Kim. 2026. Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.314

@inproceedings{monsur2026event,
  title = {{Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes}},
  author = {Abdullah {Al Monsur}, Nitesh Vamshi Bommisetty and Gene Louis Kim},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.314},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.314}
}

References

34 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ahn. 2006. The stages of event extraction. In Proceedings of the Workshop on Annotating and Reasoning about Time and Events, pages 1–8, Sydney, Australia. Association for Computational Linguistics.
  2. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization.
  3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — Qwen Technical Report
  4. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders.
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  6. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc. find
  7. Nancy Chinchor and Elaine Marsh. 1998. Appendix D: MUC-7 information extraction task definition (version 5.1). In Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax, Virginia, April 29 - May 1, 1998. find
  8. DeepSeek-AI, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi… find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. NAACL.
  10. David Dukić and Jan Snajder. 2024. Looking right is sometimes right: Investigating the capabilities of decoder-only LLMs for sequence labeling. In Findings of the Association for Computational Linguistics: ACL 2024, pages 14168–14181, Bangkok, Thailand. Association for Computational Linguistics. find
  11. Seth Ebner, Patrick Xia, Ryan Culkin, Kyle Rawlins, and Benjamin Van Durme. 2020. Multi-sentence argument linking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8057–8077, Online. Association for Computational Linguistics.
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,… — The Llama 3 Herd of Models
  13. Ralph Grishman. 1997. Information extraction: Techniques and challenges. In Information Extraction A Multidisciplinary Approach to an Emerging Information Technology, pages 10–27, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  14. Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, and Nanyun Peng. 2021. ESTER: A machine reading comprehension dataset for reasoning about event semantic relations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7543–7559, Online… find
  15. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. ICLR.
  16. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  17. Hansi Hettiarachchi and Tharindu Ranasinghe. 2023. Explainable event detection with event trigger identification as rationale extraction. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 507–518, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulga… find
  18. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
  19. Kuan-Hao Huang, I-Hung Hsu, Tanmay Parekh, Zhiyu Xie, Zixuan Zhang, Prem Natarajan, Kai-Wei Chang, Nanyun Peng, and Heng Ji. 2024. TextEE: Benchmark, reevaluation, reflections, and future challenges in event extraction. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12… find
  20. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging.
  21. Zixian Huang, Xinwei Huang, Ao Wu, Xiaxia Wang, and Gong Cheng. 2025. Transforming decoder-only models into encoder-only models with improved understanding capabilities. find
  22. Yinhan Liu et al. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  23. Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron C. Courville. 2017. Film: Visual reasoning with a general conditioning layer. volume abs/1709.07871.
  24. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xu… find
  25. Colin Raffel et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR.
  26. Dan Saattrup Nielsen, Kenneth Enevoldsen, and Peter Schneider-Kamp. 2025. Encoder vs decoder: Comparative analysis of encoder and decoder language models on multilingual NLU tasks. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Lan… find
  27. Étienne Simon, Helene Olsen, Huiling You, Samia Touileb, Lilja Øvrelid, and Erik Velldal. 2024. Generative approaches to event extraction: Survey and outlook. In Proceedings of the Workshop on the Future of Event Detection (FuturED), pages 73–86, Miami, Florida, USA. Association for Computational Li… find
  28. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958. find
  29. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Sl… find
  30. Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern,… — SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python
  31. Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020. MAVEN: A Massive General Domain Event Detection Dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1652– 1671, Onli…
  32. Zitao Wang, Xinyi Wang, and Wei Hu. 2023. Continual event extraction with semantic confusion rectification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11945–11955, Singapore. Association for Computational Linguistics. find
  33. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, et al. 2020. Transformers: State-of-the-art natural language processing. EMNLP Demos.
  34. Hongming Zhang, Xin Liu, Haojie Pan, Yangqiu Song, and Cane Wing-Ki Leung. 2020. Aser: A large-scale eventuality knowledge graph. In Proceedings of The Web Conference 2020, WWW ’20, page 201–211, New York, NY, USA. Association for Computing Machinery. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/monsur-2026-event-detection-context-aware-encoder/